-- Logs begin at Sat 2022-09-17 02:59:29 UTC, end at Sat 2022-09-17 04:07:18 UTC. -- Sep 17 03:28:32.572313 dv-cinder-849143-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 17 03:28:32.580142 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 17 03:28:32 2022] *** Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: nodename: dv-cinder-849143-3-iscsi Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: machine: x86_64 Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: clock source: unix Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: pcre jit disabled Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: detected number of CPU cores: 4 Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: current working directory: / Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: detected binary path: /usr/bin/uwsgi-core Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: your processes number limit is 47680 Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: your memory page size is 4096 bytes Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: detected max file descriptor number: 2048 Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: lock engine: pthread robust mutexes Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: thunder lock: enabled Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 17 03:28:32.586833 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 17 03:28:32.623607 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: Python main interpreter initialized at 0x55ff58e039c0 Sep 17 03:28:32.623607 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: python threads support enabled Sep 17 03:28:32.623607 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: your server socket listen backlog is limited to 100 connections Sep 17 03:28:32.623607 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: your mercy for graceful operations on workers is 90 seconds Sep 17 03:28:32.623976 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: mapped 403077 bytes (393 KB) for 2 cores Sep 17 03:28:32.624059 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: *** Operational MODE: preforking *** Sep 17 03:28:32.624293 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: *** uWSGI is running in multiple interpreter mode *** Sep 17 03:28:32.624293 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: spawned uWSGI master process (pid: 81711) Sep 17 03:28:32.624504 dv-cinder-849143-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 17 03:28:32.625774 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: spawned uWSGI worker 1 (pid: 81725, cores: 1) Sep 17 03:28:32.625774 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: spawned uWSGI worker 2 (pid: 81726, cores: 1) Sep 17 03:28:32.625774 dv-cinder-849143-3-iscsi devstack@keystone.service[81711]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 17 03:28:37.006306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 03:28:37.006306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: from cryptography.utils import int_from_bytes Sep 17 03:28:37.007110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 03:28:37.007110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: from cryptography.utils import int_from_bytes Sep 17 03:28:37.013680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 03:28:37.013680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: from cryptography.utils import int_from_bytes Sep 17 03:28:37.014242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 03:28:37.014242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: from cryptography.utils import int_from_bytes Sep 17 03:28:37.061615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 03:28:37.061815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 03:28:37.061940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] command line args: [] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 03:28:37.062218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 03:28:37.062400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ================================================================================ {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 03:28:37.062659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] admin_token = **** {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.062911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] config_dir = [] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.063102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.063286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] config_source = [] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.063470 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] debug = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.063672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.064177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.064275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.064471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.064722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.064957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] list_limit = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.065143 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] log_config_append = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.065322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.065520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] log_dir = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.065675 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] log_file = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.065939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.066197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.066443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.066690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.066870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.067049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.067295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.067542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.067779 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.068016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.068253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.068488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.068738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.068992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.069267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.069806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.069982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] publish_errors = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.070226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.070461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.070640 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.070889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.071127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.071365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] standard_threads = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.071600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.071846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.071939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 03:28:37.072028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.072103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 03:28:37.072275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] command line args: [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 03:28:37.072364 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] use_journal = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.072519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] use_json = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.072608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 03:28:37.072686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] use_stderr = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.072758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ================================================================================ {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 03:28:37.073207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] admin_token = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.073456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] config_dir = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.073643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.073846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] config_source = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.074057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] debug = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.074200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] use_syslog = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.074283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.074416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.074509 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.074620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.074702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.074910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.074983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.075242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.075381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.075496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.075580 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.075756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_config_append = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.075830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.075937 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.076015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.076115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_dir = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.076259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.076330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_file = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.076511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.076577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.076744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.076827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.076996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.077139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.077273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.077404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.077534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.077658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.077839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.077971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.078106 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.078193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.078376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.078474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.078474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.078705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.078981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.079114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.079212 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.079311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.079405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.079499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.079640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.079777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.079907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.080084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.080169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.080408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.080563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.080665 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.080762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.080902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.081137 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.081275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.081371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.081443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] publish_errors = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.081612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.081686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.081889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.081969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.082100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.082172 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.082347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.082422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.082582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.082648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.082817 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] standard_threads = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.082890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.083051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.083128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.083436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.083516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.083620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.083699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.083770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] use_journal = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.083839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.083900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] use_json = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.083991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.084057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] use_stderr = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.084209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] use_syslog = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.084274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.084449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.084750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.084916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.085149 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.085390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.085641 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.085756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 03:28:37.085958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.086031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.086155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.086256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.086391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.086533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.086631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.086761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.086968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.087128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.087348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.087589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.087832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.088120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.088207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.088315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.088388 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.088623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.088790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.088856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.089165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.089285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.089523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.089776 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.090043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.090169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.090285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.090482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.090568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.090806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.090878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.091047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.091113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.091310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.091386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.091723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.091794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.091860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.091957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.092076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.092185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.092287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.092395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.092490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.092589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.092743 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.092868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.092966 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.093127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.093262 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.093409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.093510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.093620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.093763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.093882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.093989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.094122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.094234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.094388 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.094503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.094602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.094701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.094842 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.094946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.095045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.095150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.095287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.095390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.095548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.095652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.095778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.095917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.096132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.096341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.096481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.096595 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.096707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.096829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.096905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.096966 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.097073 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.097157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.097259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.097380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.097467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.097617 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.097797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.097909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.098098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.098328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.098415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.098548 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.098651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.098780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.098857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.098937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.099010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.099133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.099213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.099328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.099406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.099655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.099735 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.099891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.099970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.100131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.100363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.100601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.100839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.101088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.101225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.101309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.101513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.101601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.101801 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] auth.external = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.101918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.103203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.103299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.103395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.103474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.103567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.103643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.103713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.103784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] auth.password = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.103867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.103935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] auth.token = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.104018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.104098 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.104170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.104246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.104320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.104402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.104483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.104584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.104778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.104903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.105027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.105129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.105271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.105407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.105622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.105716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] credential.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.105814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.105887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.111229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.111411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.111487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.111558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.111629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.111700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.111783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.111863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.111941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.112952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113477 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] federation.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.113956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114024 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.114960 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.115942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116009 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116265 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116384 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.external = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116900 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.116961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.password = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.token = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.117979 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118667 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.118931 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.119013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.119088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.119152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.119214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.119384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.119507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.119617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.119714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.119841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.119936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.120102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.120189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.120350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.120432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.120585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.120665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.120829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.120909 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.121066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.121308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.121549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.122477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.122706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.122943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.123048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.123217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.123298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.123463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.123667 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.123745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.123909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.123978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.124147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.124215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.124403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.124483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.124653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.124807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.124896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.125152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.125390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.125647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.125907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.126037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.126164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.126338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.126442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.126608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.126715 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.126861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.126965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.127085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.127160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.127321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.127409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.127568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.127646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.127800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.127878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.128037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.128180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.128288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.128359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.128546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.128687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.128825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.129004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.129171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.129351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.129490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.129623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.129623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.129792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.130018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.130134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.130270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.130400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.130524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.130655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.130776 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.131028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.131253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.131332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.131504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.131601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.131748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.131832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.131988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.132065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.132230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.132309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.132471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.132548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.132722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.132800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.132957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.133037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.133200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.133285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.133449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.133527 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.133683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.133818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.133960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.134039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.134193 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.134307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.139903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] resource.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140628 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140698 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.140989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141059 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] role.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141661 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] role.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.141991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.142294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.142374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.142447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.142518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.142588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.142659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.142729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.142807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.142880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.143991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144360 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144877 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.144948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.145942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.146019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.146091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.146091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.146211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.146358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.146430 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.146610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.146678 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.146845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.146912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.147090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] token.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.147159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.147403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.147470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.147695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.147807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.147951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.148054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.148193 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.148330 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.148430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.148612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.148691 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.148880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.148957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.149128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.149195 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.149404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.149510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.149671 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.149790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.150034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.150148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.150251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.150474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.150561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.150733 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.150907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.150989 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.151203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.151369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.151599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.151898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.151997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.152075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.152164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.152308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.152471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.152604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.152731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.152861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.152990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.connection = **** {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.153126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.153321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.153571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.153712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.154060 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.154200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.154323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.154508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.154632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.154765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.154981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.155104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.155229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] role.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.155419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.155557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] role.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.155866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.156126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.156249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.156375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.156510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.156636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.156761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.156886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.157011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.157138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.157273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.157398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.157479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.157708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.157866 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.157958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.158034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.158167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.158246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.158419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.158498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.158673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.158761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.158918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.158996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.159163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.159241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.159342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 03:28:37.159592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.159833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.160083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.160340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.160590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.160842 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.161079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.161335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.161586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.161852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.162086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.162320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.162551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.162803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.163058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.163308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.163545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.163820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.164056 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.164402 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.164642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.164890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.165128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.165385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.165634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.165900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.166147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.166394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.166643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.166881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.167120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.167358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.167605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.167885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.168117 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.168362 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.169389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.169522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.connection = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.169834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.170268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.170550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.170803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.171044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.171286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.171525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.171764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.172009 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.172247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.173660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.173660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.173660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.173660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.173660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.173660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.180223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.180223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 03:28:37.180223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 03:28:37.203082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.203310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.203535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.207475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.207657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.207883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.217299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.217435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.248069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.248300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.248490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.248728 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.318496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.322759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.374821 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.375048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.381078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.381316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.537967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.538241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.538417 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.538736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.538955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.539172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.555248 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.555691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.556039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.556432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.556829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.557224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.565812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.566049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.566278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.566498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.566713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.566921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.567141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.579269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.581499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.581756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.581989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.582212 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.582432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.582652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.582870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.592040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.593052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.593282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.593501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.593719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.593957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.594169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.594381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.595182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.595423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.595645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.595865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.596083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.596299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.596514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81726) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 03:28:37.615870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.616080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 03:28:37.616423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.616621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 03:28:37.616890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.617148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.617403 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.617663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.619866 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.620077 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 03:28:37.620424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.620627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 03:28:37.620903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.621165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.621429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.621867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.627459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.629825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.629825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.629825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.631709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.631994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.632270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.632555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.635205 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.638689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.638967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.639251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.639534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.639814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.640070 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.640337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.640417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.640609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.640878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.641150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.641527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.643321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.643613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.643903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.644188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.644474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.644737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.645010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.645287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.645810 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.646316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.646840 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.664321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.665806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.666090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.671332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.671612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.671890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.678277 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.678552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.680457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.681965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.682252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.684320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.684618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.685041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.685331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.685611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.685914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.686185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.686453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.687283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.687565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.687849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.694200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.694491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.701923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.702456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.702967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.703329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.703469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.703610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.703963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.704460 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.704957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.705452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.707473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.710717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.711011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.715214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.718615 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.721919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.722214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.722498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.722783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.723077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.723357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.726473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.726974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.735264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.738700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.739003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.739886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.740195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.740498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.740760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.743251 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.746432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.747949 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.749478 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.750150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.750455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.750745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.751022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.751105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.751297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.751382 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.751613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.753993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.754278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.754558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.758963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.758963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.766991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.767276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.767559 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.767841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.768221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.768502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.768781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.771977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.771977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.771977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.772243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.779479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.781043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.782599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.782883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.784389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.785532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.785854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.786141 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.786439 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.786831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.787408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.790341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.791792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.793256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.794425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.794721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.798928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.799241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.799506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.799790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.800176 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.800460 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.801116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.801209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.802568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.805599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.806027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.806317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.806595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.815136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.815434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.815713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.817122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.820053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.822695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.823232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.823323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.823518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.823794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.824067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.824338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.824611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.824692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.824922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.825185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.825466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.825868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.826135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.826463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.826588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.826799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.826881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.827278 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.839317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.839317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.839317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.839317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.839317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.839317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.847427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.847723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.848007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.854972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81726) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 03:28:37.855265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.855546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.855824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.856724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.857216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.857506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.857811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.858091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.858366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.858656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.858932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.859201 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.859473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81726) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 03:28:37.861703 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ff58e039c0 pid: 81725 (default app) Sep 17 03:28:37.865032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e096f-48d1-4442-b48c-ca402dd86de7 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:37.865315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e096f-48d1-4442-b48c-ca402dd86de7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:37.865589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e096f-48d1-4442-b48c-ca402dd86de7 None None] PATH_INFO: `/v3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:37.869897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Sat Sep 17 03:28:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:28:37.896566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ff58e039c0 pid: 81726 (default app) Sep 17 03:28:45.728908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f1e09-d9e9-4c0c-9d47-d48f37d4b4ad None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:45.729439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f1e09-d9e9-4c0c-9d47-d48f37d4b4ad None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:45.729439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f1e09-d9e9-4c0c-9d47-d48f37d4b4ad None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:45.731503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 2/2] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:28:45.739717 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:45.740158 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:45.740253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:45.812064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: INFO dbcounter [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] Registered counter for database keystone Sep 17 03:28:45.832362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG oslo_db.sqlalchemy.engines [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81726) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 03:28:45.838512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writer thread running {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 03:28:46.029194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG passlib.handlers.bcrypt [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81726) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 17 03:28:46.029510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG passlib.handlers.bcrypt [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81726) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 17 03:28:46.088426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:46.090042 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:46.221993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1/3] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 17 03:28:46.680666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f1e09-d9e9-4c0c-9d47-d48f37d4b4ad None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:46.763143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: INFO dbcounter [None req-031f1e09-d9e9-4c0c-9d47-d48f37d4b4ad None None] Registered counter for database keystone Sep 17 03:28:46.784363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG oslo_db.sqlalchemy.engines [None req-031f1e09-d9e9-4c0c-9d47-d48f37d4b4ad None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81725) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 03:28:46.790513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writer thread running {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 03:28:46.917250 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f1e09-d9e9-4c0c-9d47-d48f37d4b4ad None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:46.921898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3000c-980b-4d56-8c26-1789a062bcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:46.922526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3000c-980b-4d56-8c26-1789a062bcef admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:46.922809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3000c-980b-4d56-8c26-1789a062bcef admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:46.923081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3000c-980b-4d56-8c26-1789a062bcef admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:46.931275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3000c-980b-4d56-8c26-1789a062bcef admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:46.984213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2da3000c-980b-4d56-8c26-1789a062bcef admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 17 03:28:46.984558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3000c-980b-4d56-8c26-1789a062bcef admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:46.993859 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 3/4] 10.222.0.42 () {58 vars in 1324 bytes} [Sat Sep 17 03:28:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:28:47.000571 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:47.016242 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b064-b69e-40da-ae8c-25a85ae39bf6 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:47.020654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eaa01f-6c0e-4d4e-80db-942ec5d834b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:47.021805 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaa01f-6c0e-4d4e-80db-942ec5d834b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:47.022058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaa01f-6c0e-4d4e-80db-942ec5d834b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:47.022334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaa01f-6c0e-4d4e-80db-942ec5d834b0 admin admin] PATH_INFO: `/v3/projects/545b5824a9314a3a81e4babed059fa06` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:47.023768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eaa01f-6c0e-4d4e-80db-942ec5d834b0 admin admin] RBAC: Authorizing `identity:get_project(project_id=545b5824a9314a3a81e4babed059fa06)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:47.077268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: WARNING keystone.common.rbac_enforcer.enforcer [None req-17eaa01f-6c0e-4d4e-80db-942ec5d834b0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 17 03:28:47.077611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eaa01f-6c0e-4d4e-80db-942ec5d834b0 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:47.082132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 2/5] 10.222.0.42 () {56 vars in 1207 bytes} [Sat Sep 17 03:28:46 2022] GET /identity/v3/projects/545b5824a9314a3a81e4babed059fa06 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:28:49.184158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5f81e-2a08-409c-88fa-23c39dd67ee3 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:49.184708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5f81e-2a08-409c-88fa-23c39dd67ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:49.184708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5f81e-2a08-409c-88fa-23c39dd67ee3 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:49.188865 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 4/6] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:28:49.194279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df2459-8b2b-4cd8-a725-18b90499dceb None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:49.194681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df2459-8b2b-4cd8-a725-18b90499dceb None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:49.194834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df2459-8b2b-4cd8-a725-18b90499dceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:49.214858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-c1df2459-8b2b-4cd8-a725-18b90499dceb None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:49.216883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c1df2459-8b2b-4cd8-a725-18b90499dceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:49.224589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 3/7] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 17 03:28:49.490157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5f81e-2a08-409c-88fa-23c39dd67ee3 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:49.502681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5f81e-2a08-409c-88fa-23c39dd67ee3 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:49.506875 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4545cd-d714-42fe-abf1-578d637564f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:49.507471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4545cd-d714-42fe-abf1-578d637564f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:49.507754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4545cd-d714-42fe-abf1-578d637564f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:49.508029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4545cd-d714-42fe-abf1-578d637564f7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:49.516341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4545cd-d714-42fe-abf1-578d637564f7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:49.518142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4545cd-d714-42fe-abf1-578d637564f7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:49.526795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 5/8] 10.222.0.42 () {58 vars in 1324 bytes} [Sat Sep 17 03:28:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:28:49.532044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df2459-8b2b-4cd8-a725-18b90499dceb None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:49.541004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df2459-8b2b-4cd8-a725-18b90499dceb None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:49.545594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3eb3b-572f-46ba-ac37-b1b86aade6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:49.546244 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3eb3b-572f-46ba-ac37-b1b86aade6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:49.546522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3eb3b-572f-46ba-ac37-b1b86aade6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:49.546792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3eb3b-572f-46ba-ac37-b1b86aade6f1 admin admin] PATH_INFO: `/v3/users/bb19e7f52e3f4b85bcc7952e23773462` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:49.548083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3eb3b-572f-46ba-ac37-b1b86aade6f1 admin admin] RBAC: Authorizing `identity:get_user(user_id=bb19e7f52e3f4b85bcc7952e23773462)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:49.549929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3eb3b-572f-46ba-ac37-b1b86aade6f1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:49.551429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 4/9] 10.222.0.42 () {56 vars in 1198 bytes} [Sat Sep 17 03:28:49 2022] GET /identity/v3/users/bb19e7f52e3f4b85bcc7952e23773462 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:28:51.770554 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14d4cf-6e76-4ce1-a6d1-f4f2ac2de8b2 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:51.771169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14d4cf-6e76-4ce1-a6d1-f4f2ac2de8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:51.771169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14d4cf-6e76-4ce1-a6d1-f4f2ac2de8b2 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:51.772484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 6/10] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:28:51.778131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee3acc-c48b-48f6-82c4-4465ef6bc730 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:51.778735 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee3acc-c48b-48f6-82c4-4465ef6bc730 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:51.778735 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee3acc-c48b-48f6-82c4-4465ef6bc730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:51.798774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-78ee3acc-c48b-48f6-82c4-4465ef6bc730 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:51.798774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-78ee3acc-c48b-48f6-82c4-4465ef6bc730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:51.819399 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaca65-2782-4f48-88fa-12a424bada4e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:51.819682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaca65-2782-4f48-88fa-12a424bada4e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:51.819953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaca65-2782-4f48-88fa-12a424bada4e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:51.821314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 7/11] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:28:51.827392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5acd7c-4dbf-4db6-a83a-1b6a3a822c23 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:51.827738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5acd7c-4dbf-4db6-a83a-1b6a3a822c23 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:51.828078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5acd7c-4dbf-4db6-a83a-1b6a3a822c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:51.870200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG passlib.handlers.bcrypt [None req-0e5acd7c-4dbf-4db6-a83a-1b6a3a822c23 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81725) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 17 03:28:51.870593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG passlib.handlers.bcrypt [None req-0e5acd7c-4dbf-4db6-a83a-1b6a3a822c23 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81725) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 17 03:28:51.878417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 5/12] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:28:51.921000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0e5acd7c-4dbf-4db6-a83a-1b6a3a822c23 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:51.922666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0e5acd7c-4dbf-4db6-a83a-1b6a3a822c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:52.021961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 8/13] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:28:52.146626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee3acc-c48b-48f6-82c4-4465ef6bc730 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:52.170205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:52.249450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee3acc-c48b-48f6-82c4-4465ef6bc730 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:52.283676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037918b-b1d6-4b1a-bf95-8d20b41c6637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:52.284240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037918b-b1d6-4b1a-bf95-8d20b41c6637 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:52.284522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037918b-b1d6-4b1a-bf95-8d20b41c6637 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:52.284796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037918b-b1d6-4b1a-bf95-8d20b41c6637 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:52.293385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5acd7c-4dbf-4db6-a83a-1b6a3a822c23 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:52.296556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037918b-b1d6-4b1a-bf95-8d20b41c6637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-e037918b-b1d6-4b1a-bf95-8d20b41c6637 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:28:52.305047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:28:52.306034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:28:52.307004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 6/14] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:28:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:28:52.312387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037918b-b1d6-4b1a-bf95-8d20b41c6637 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:52.393191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:52.396172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5acd7c-4dbf-4db6-a83a-1b6a3a822c23 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:52.398536 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:52.410692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037918b-b1d6-4b1a-bf95-8d20b41c6637 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:52.433035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7c4fd-d76c-4eea-b4cc-be6bb6928224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:52.434435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7c4fd-d76c-4eea-b4cc-be6bb6928224 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:52.434713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7c4fd-d76c-4eea-b4cc-be6bb6928224 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:52.435007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7c4fd-d76c-4eea-b4cc-be6bb6928224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:52.444574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93047a90-2559-4996-9829-496c439bfd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:52.445181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93047a90-2559-4996-9829-496c439bfd5e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:52.445469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93047a90-2559-4996-9829-496c439bfd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:52.445763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93047a90-2559-4996-9829-496c439bfd5e None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:52.446456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93047a90-2559-4996-9829-496c439bfd5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:28:52.446736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93047a90-2559-4996-9829-496c439bfd5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:52.448451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93047a90-2559-4996-9829-496c439bfd5e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:52.460319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 7/15] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:28:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:28:52.466379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93047a90-2559-4996-9829-496c439bfd5e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:52.503455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7c4fd-d76c-4eea-b4cc-be6bb6928224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:52.504970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7c4fd-d76c-4eea-b4cc-be6bb6928224 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:52.564346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93047a90-2559-4996-9829-496c439bfd5e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:52.597645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc29677-b407-46a0-a7be-d50015ba2c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:52.598290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc29677-b407-46a0-a7be-d50015ba2c76 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:52.598568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc29677-b407-46a0-a7be-d50015ba2c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:52.598837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc29677-b407-46a0-a7be-d50015ba2c76 None admin] PATH_INFO: `/v3/users/bb19e7f52e3f4b85bcc7952e23773462` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:52.599976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc29677-b407-46a0-a7be-d50015ba2c76 None admin] RBAC: Authorizing `identity:get_user(user_id=bb19e7f52e3f4b85bcc7952e23773462)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:52.601417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc29677-b407-46a0-a7be-d50015ba2c76 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:52.601626 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 9/16] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 03:28:52.607940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 8/17] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:28:52 2022] GET /identity/v3/users/bb19e7f52e3f4b85bcc7952e23773462 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:28:52.608087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7c4fd-d76c-4eea-b4cc-be6bb6928224 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:52.609132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc29677-b407-46a0-a7be-d50015ba2c76 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:52.610595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:52.624373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:52.706105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7c4fd-d76c-4eea-b4cc-be6bb6928224 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:52.707089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc29677-b407-46a0-a7be-d50015ba2c76 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:52.741048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b706d6-c38e-481e-aa83-0c641781024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:52.741245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd0ba0-8ff2-404d-9b55-f638a6f07bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:52.741764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd0ba0-8ff2-404d-9b55-f638a6f07bae None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:52.741927 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b706d6-c38e-481e-aa83-0c641781024a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:52.742045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd0ba0-8ff2-404d-9b55-f638a6f07bae None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:52.742201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b706d6-c38e-481e-aa83-0c641781024a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:52.742322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd0ba0-8ff2-404d-9b55-f638a6f07bae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:52.742469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b706d6-c38e-481e-aa83-0c641781024a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:52.743461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd0ba0-8ff2-404d-9b55-f638a6f07bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:52.744919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd0ba0-8ff2-404d-9b55-f638a6f07bae None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:52.747070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:28:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:28:52.753110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd0ba0-8ff2-404d-9b55-f638a6f07bae None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:52.762148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b706d6-c38e-481e-aa83-0c641781024a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:52.764845 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b706d6-c38e-481e-aa83-0c641781024a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-05b706d6-c38e-481e-aa83-0c641781024a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:28:52.784888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:28:52.786311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:28:52.787161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:28:52.787161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:28:52.787161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:28:52.787161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:28:52.791851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b706d6-c38e-481e-aa83-0c641781024a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:52.843080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:52.851009 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:52.861630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd0ba0-8ff2-404d-9b55-f638a6f07bae None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:52.894395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b706d6-c38e-481e-aa83-0c641781024a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:52.896054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7711a-6d85-4f9f-86c5-22d6852b94e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:52.896662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7711a-6d85-4f9f-86c5-22d6852b94e1 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:52.896940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7711a-6d85-4f9f-86c5-22d6852b94e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:52.897211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7711a-6d85-4f9f-86c5-22d6852b94e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:52.897960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7711a-6d85-4f9f-86c5-22d6852b94e1 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.domain.id=default, user.id=bb19e7f52e3f4b85bcc7952e23773462) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:28:52.898401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7711a-6d85-4f9f-86c5-22d6852b94e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:52.899729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7711a-6d85-4f9f-86c5-22d6852b94e1 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:52.907113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1322 bytes} [Sat Sep 17 03:28:52 2022] GET /identity/v3/role_assignments?user.id=bb19e7f52e3f4b85bcc7952e23773462&scope.domain.id=default&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:28:52.929922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81739f5e-ae82-46da-a616-967ee971eaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:52.930547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81739f5e-ae82-46da-a616-967ee971eaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:52.930828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81739f5e-ae82-46da-a616-967ee971eaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:52.931100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81739f5e-ae82-46da-a616-967ee971eaf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:52.931794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81739f5e-ae82-46da-a616-967ee971eaf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:28:52.932097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81739f5e-ae82-46da-a616-967ee971eaf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:52.933546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81739f5e-ae82-46da-a616-967ee971eaf6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:52.953562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 11/21] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:28:55.194609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c8187-3a4b-412d-a930-4eabf0ce6cde None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:55.195119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c8187-3a4b-412d-a930-4eabf0ce6cde None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:55.195220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c8187-3a4b-412d-a930-4eabf0ce6cde None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:55.196468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:28:55.202166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31d0b1-7778-4ce1-a49d-26f2a6df2308 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:55.202576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31d0b1-7778-4ce1-a49d-26f2a6df2308 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:55.202894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31d0b1-7778-4ce1-a49d-26f2a6df2308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:55.222122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3e31d0b1-7778-4ce1-a49d-26f2a6df2308 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:55.223409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3e31d0b1-7778-4ce1-a49d-26f2a6df2308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:55.315335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 12/23] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:28:55.499967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66fc23-3e15-458a-93d5-07183fd2abde None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:55.500145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66fc23-3e15-458a-93d5-07183fd2abde None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:55.500441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66fc23-3e15-458a-93d5-07183fd2abde None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:55.502062 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 12/24] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:28:55.508168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477b91e-12e2-4050-8cbf-a08498005455 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:55.508490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477b91e-12e2-4050-8cbf-a08498005455 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:55.508723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477b91e-12e2-4050-8cbf-a08498005455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:55.538767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5477b91e-12e2-4050-8cbf-a08498005455 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:55.540035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5477b91e-12e2-4050-8cbf-a08498005455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:55.597156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:55.623118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 13/25] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:28:55.896686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66fc23-3e15-458a-93d5-07183fd2abde None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:55.915375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477b91e-12e2-4050-8cbf-a08498005455 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:56.075854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66fc23-3e15-458a-93d5-07183fd2abde None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:56.133635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477b91e-12e2-4050-8cbf-a08498005455 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:56.160083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b209e-c630-4f67-8bb0-ef0b3085f5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:56.160736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b209e-c630-4f67-8bb0-ef0b3085f5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.161213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b209e-c630-4f67-8bb0-ef0b3085f5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.161631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b209e-c630-4f67-8bb0-ef0b3085f5c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.190599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b209e-c630-4f67-8bb0-ef0b3085f5c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:56.192353 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b209e-c630-4f67-8bb0-ef0b3085f5c8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:56.199151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:56.201717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fd8e8-2ec1-4f09-9f57-386620bdf344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:56.202407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fd8e8-2ec1-4f09-9f57-386620bdf344 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.202903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fd8e8-2ec1-4f09-9f57-386620bdf344 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.202999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fd8e8-2ec1-4f09-9f57-386620bdf344 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.221310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fd8e8-2ec1-4f09-9f57-386620bdf344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d38b209e-c630-4f67-8bb0-ef0b3085f5c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:28:56.222416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:28:56.223457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:28:56.224486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:28:56.224486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:28:56.224486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-473fd8e8-2ec1-4f09-9f57-386620bdf344 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:28:56.229783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:28:56.231791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:28:56.232870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:28:56.234061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 14/26] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:28:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:28:56.235009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 13/27] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:28:56.239976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fd8e8-2ec1-4f09-9f57-386620bdf344 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:56.241983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b209e-c630-4f67-8bb0-ef0b3085f5c8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:56.330248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:56.421214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fd8e8-2ec1-4f09-9f57-386620bdf344 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:56.469802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b209e-c630-4f67-8bb0-ef0b3085f5c8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:56.471394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c893cb8-52b3-446a-bad6-dc55ee9c0dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:56.472002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c893cb8-52b3-446a-bad6-dc55ee9c0dcf None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.472288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c893cb8-52b3-446a-bad6-dc55ee9c0dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.472569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c893cb8-52b3-446a-bad6-dc55ee9c0dcf None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.473256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c893cb8-52b3-446a-bad6-dc55ee9c0dcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:28:56.473563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c893cb8-52b3-446a-bad6-dc55ee9c0dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:56.475002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c893cb8-52b3-446a-bad6-dc55ee9c0dcf None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:56.495977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 15/28] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:28:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:28:56.497604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f2a4-b9d4-487a-be4c-0244980bb1ab None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.497918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f2a4-b9d4-487a-be4c-0244980bb1ab None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.498189 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f2a4-b9d4-487a-be4c-0244980bb1ab None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.499577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 16/29] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:28:56.504634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9f2a4-b9d4-487a-be4c-0244980bb1ab None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:56.519773 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff74cb-9ab4-4dc0-98bd-b547df2826d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:56.520386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff74cb-9ab4-4dc0-98bd-b547df2826d4 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.520669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff74cb-9ab4-4dc0-98bd-b547df2826d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.520948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff74cb-9ab4-4dc0-98bd-b547df2826d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.521605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff74cb-9ab4-4dc0-98bd-b547df2826d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:28:56.521940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff74cb-9ab4-4dc0-98bd-b547df2826d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:56.523334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff74cb-9ab4-4dc0-98bd-b547df2826d4 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:56.541943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 14/30] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:28:56.543802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138bcaa-2b0e-43d1-96a6-3050b9871444 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.544160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138bcaa-2b0e-43d1-96a6-3050b9871444 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.545297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138bcaa-2b0e-43d1-96a6-3050b9871444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.566162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a138bcaa-2b0e-43d1-96a6-3050b9871444 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:56.567411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a138bcaa-2b0e-43d1-96a6-3050b9871444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:56.615031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:56.619712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9f2a4-b9d4-487a-be4c-0244980bb1ab None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:56.640522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:56.659359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6dea2-299f-4ece-84d6-85a79c4b247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:56.660176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6dea2-299f-4ece-84d6-85a79c4b247e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.660632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6dea2-299f-4ece-84d6-85a79c4b247e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.661050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6dea2-299f-4ece-84d6-85a79c4b247e None admin] PATH_INFO: `/v3/users/bb19e7f52e3f4b85bcc7952e23773462` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.662847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 15/31] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:28:56.664216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6dea2-299f-4ece-84d6-85a79c4b247e None admin] RBAC: Authorizing `identity:get_user(user_id=bb19e7f52e3f4b85bcc7952e23773462)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:56.664878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6dea2-299f-4ece-84d6-85a79c4b247e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:56.665871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcea230-5a9f-4ff1-9871-488109ba42d8 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.667909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcea230-5a9f-4ff1-9871-488109ba42d8 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.668960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcea230-5a9f-4ff1-9871-488109ba42d8 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.670550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 16/33] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:28:56.674425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 17/33] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:28:56 2022] GET /identity/v3/users/bb19e7f52e3f4b85bcc7952e23773462 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:28:56.674425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6dea2-299f-4ece-84d6-85a79c4b247e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:56.675546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcea230-5a9f-4ff1-9871-488109ba42d8 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:56.831235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6dea2-299f-4ece-84d6-85a79c4b247e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:56.874115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcea230-5a9f-4ff1-9871-488109ba42d8 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:56.877380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e48c9-64cd-4ad8-8892-baa0cfc06f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:56.878026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e48c9-64cd-4ad8-8892-baa0cfc06f8f None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.878311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e48c9-64cd-4ad8-8892-baa0cfc06f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.878589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e48c9-64cd-4ad8-8892-baa0cfc06f8f None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.879252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e48c9-64cd-4ad8-8892-baa0cfc06f8f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:56.880712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e48c9-64cd-4ad8-8892-baa0cfc06f8f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:56.929193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99d4d7-406c-45b7-918d-b0b1d8622354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:56.929864 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99d4d7-406c-45b7-918d-b0b1d8622354 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.930202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99d4d7-406c-45b7-918d-b0b1d8622354 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.930483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99d4d7-406c-45b7-918d-b0b1d8622354 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.931610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99d4d7-406c-45b7-918d-b0b1d8622354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:56.933184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99d4d7-406c-45b7-918d-b0b1d8622354 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:56.935367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 17/34] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:28:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:28:56.940150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3407a5-c00b-4576-98a6-ebb29856b165 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:56.940150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3407a5-c00b-4576-98a6-ebb29856b165 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:56.940150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3407a5-c00b-4576-98a6-ebb29856b165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:56.962991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-2b3407a5-c00b-4576-98a6-ebb29856b165 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:56.964350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-2b3407a5-c00b-4576-98a6-ebb29856b165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:57.040081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:57.060427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 18/35] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:28:57.061270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3407a5-c00b-4576-98a6-ebb29856b165 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:57.141441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 18/36] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:28:56 2022] POST /identity/v3/projects => generated 301 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:28:57.142429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e48c9-64cd-4ad8-8892-baa0cfc06f8f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:57.172101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:57.190660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3407a5-c00b-4576-98a6-ebb29856b165 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:57.229397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d78215-a5c2-4ec9-876b-8418db434027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:57.230157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78215-a5c2-4ec9-876b-8418db434027 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:57.230382 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78215-a5c2-4ec9-876b-8418db434027 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:57.231367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78215-a5c2-4ec9-876b-8418db434027 None admin] PATH_INFO: `/v3/domains/default/users/bb19e7f52e3f4b85bcc7952e23773462/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:57.233449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d78215-a5c2-4ec9-876b-8418db434027 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bb19e7f52e3f4b85bcc7952e23773462, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:57.234977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d78215-a5c2-4ec9-876b-8418db434027 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:57.274044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e48c9-64cd-4ad8-8892-baa0cfc06f8f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:57.313204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb6cd9-a49d-4ee9-9daf-7a6307a7e01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:57.313892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb6cd9-a49d-4ee9-9daf-7a6307a7e01f None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:57.314183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb6cd9-a49d-4ee9-9daf-7a6307a7e01f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:57.314465 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb6cd9-a49d-4ee9-9daf-7a6307a7e01f None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:57.315166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb6cd9-a49d-4ee9-9daf-7a6307a7e01f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:57.316681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb6cd9-a49d-4ee9-9daf-7a6307a7e01f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:57.390664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 19/37] 10.222.0.42 () {60 vars in 1131 bytes} [Sat Sep 17 03:28:57 2022] POST /identity/v3/roles => generated 219 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:28:57.392066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260b77c-f2a6-48b7-af9a-e1b594914eb9 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:57.392325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260b77c-f2a6-48b7-af9a-e1b594914eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:57.392605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260b77c-f2a6-48b7-af9a-e1b594914eb9 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:57.393894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 20/38] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:28:57.394936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbc527-092e-48b0-b2a2-631795f8efbf None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:57.395221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbc527-092e-48b0-b2a2-631795f8efbf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:57.395497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbc527-092e-48b0-b2a2-631795f8efbf None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:57.396763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 21/39] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:28:57.397491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbc527-092e-48b0-b2a2-631795f8efbf None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:57.440639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 19/40] 10.222.0.42 () {58 vars in 1361 bytes} [Sat Sep 17 03:28:57 2022] PUT /identity/v3/domains/default/users/bb19e7f52e3f4b85bcc7952e23773462/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:28:57.442210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78d553-0af9-4cda-8397-3491e9b65e67 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:57.442511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78d553-0af9-4cda-8397-3491e9b65e67 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:57.442794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78d553-0af9-4cda-8397-3491e9b65e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:57.467919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-0d78d553-0af9-4cda-8397-3491e9b65e67 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:57.469897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0d78d553-0af9-4cda-8397-3491e9b65e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:57.506545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:57.538271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbc527-092e-48b0-b2a2-631795f8efbf None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:57.546268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:57.581824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 20/41] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:28:57.581824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed270-b85b-4e0b-96ad-4286f9a4b6c6 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:57.581824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed270-b85b-4e0b-96ad-4286f9a4b6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:57.581824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed270-b85b-4e0b-96ad-4286f9a4b6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:57.599687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440f094-3a8e-4000-b605-2aca52b6951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:57.602144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440f094-3a8e-4000-b605-2aca52b6951a None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:57.602357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440f094-3a8e-4000-b605-2aca52b6951a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:57.602587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440f094-3a8e-4000-b605-2aca52b6951a None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:57.603319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440f094-3a8e-4000-b605-2aca52b6951a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:57.606515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440f094-3a8e-4000-b605-2aca52b6951a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:57.611922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-070ed270-b85b-4e0b-96ad-4286f9a4b6c6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:57.613461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-070ed270-b85b-4e0b-96ad-4286f9a4b6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:57.697034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 22/42] 10.222.0.42 () {60 vars in 1131 bytes} [Sat Sep 17 03:28:57 2022] POST /identity/v3/roles => generated 225 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:28:57.700707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 21/43] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:28:57.875113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440f094-3a8e-4000-b605-2aca52b6951a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:57.975853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ed270-b85b-4e0b-96ad-4286f9a4b6c6 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:57.987627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440f094-3a8e-4000-b605-2aca52b6951a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:58.027129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067d056-1d50-4c68-aa26-a611328e7d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:58.028514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067d056-1d50-4c68-aa26-a611328e7d5e None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:58.028914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067d056-1d50-4c68-aa26-a611328e7d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:58.029128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067d056-1d50-4c68-aa26-a611328e7d5e None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:58.029827 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067d056-1d50-4c68-aa26-a611328e7d5e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:58.031229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067d056-1d50-4c68-aa26-a611328e7d5e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:58.082917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ed270-b85b-4e0b-96ad-4286f9a4b6c6 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:58.085197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:58.096679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:58.103397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 23/44] 10.222.0.42 () {60 vars in 1131 bytes} [Sat Sep 17 03:28:57 2022] POST /identity/v3/roles => generated 223 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:28:58.229302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3561a6-3fb6-4ee0-a048-50a5227022da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:58.229896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3561a6-3fb6-4ee0-a048-50a5227022da None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:58.230445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3561a6-3fb6-4ee0-a048-50a5227022da None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:58.230445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3561a6-3fb6-4ee0-a048-50a5227022da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:58.231723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3561a6-3fb6-4ee0-a048-50a5227022da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:58.233433 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3561a6-3fb6-4ee0-a048-50a5227022da None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:58.238277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 22/45] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:28:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:28:58.243994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067d056-1d50-4c68-aa26-a611328e7d5e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:28:58.354760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067d056-1d50-4c68-aa26-a611328e7d5e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:28:58.391765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c9f9b-6eef-47f4-9bd1-cc308ba17cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:28:58.392367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c9f9b-6eef-47f4-9bd1-cc308ba17cb7 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:58.392649 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c9f9b-6eef-47f4-9bd1-cc308ba17cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:58.392925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c9f9b-6eef-47f4-9bd1-cc308ba17cb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:58.393582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c9f9b-6eef-47f4-9bd1-cc308ba17cb7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:28:58.395054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c9f9b-6eef-47f4-9bd1-cc308ba17cb7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:28:58.603510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:28:58 2022] POST /identity/v3/projects => generated 312 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:28:59.725401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e5ec5-cf85-40c8-8234-7f063f0f1178 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:59.725945 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e5ec5-cf85-40c8-8234-7f063f0f1178 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:59.726025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e5ec5-cf85-40c8-8234-7f063f0f1178 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:59.727318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 23/47] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:28:59.733024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63edbae-787f-45ff-a22d-480d6aec6425 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:28:59.733443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63edbae-787f-45ff-a22d-480d6aec6425 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:28:59.733586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63edbae-787f-45ff-a22d-480d6aec6425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:28:59.752272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a63edbae-787f-45ff-a22d-480d6aec6425 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:28:59.753789 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a63edbae-787f-45ff-a22d-480d6aec6425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:28:59.801960 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:28:59.822486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 25/48] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:00.090695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e5ec5-cf85-40c8-8234-7f063f0f1178 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:00.194021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e5ec5-cf85-40c8-8234-7f063f0f1178 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:00.226427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d31eb-bdb1-4600-8447-65de807d0eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:00.227051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d31eb-bdb1-4600-8447-65de807d0eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:00.227329 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d31eb-bdb1-4600-8447-65de807d0eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:00.227602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d31eb-bdb1-4600-8447-65de807d0eb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:00.238015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d31eb-bdb1-4600-8447-65de807d0eb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-445d31eb-bdb1-4600-8447-65de807d0eb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:00.242833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:00.243774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:00.244790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 24/49] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:29:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:00.250378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63edbae-787f-45ff-a22d-480d6aec6425 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:00.345525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63edbae-787f-45ff-a22d-480d6aec6425 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:00.379041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52a247-e955-470b-91f7-b25bc6705f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:00.379647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a247-e955-470b-91f7-b25bc6705f75 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:00.379929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a247-e955-470b-91f7-b25bc6705f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:00.380203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a247-e955-470b-91f7-b25bc6705f75 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:00.380862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52a247-e955-470b-91f7-b25bc6705f75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:00.381165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52a247-e955-470b-91f7-b25bc6705f75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:00.382569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52a247-e955-470b-91f7-b25bc6705f75 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:00.390633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 26/50] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:00.396786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d31eb-bdb1-4600-8447-65de807d0eb6 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:00.427892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:00.496614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d31eb-bdb1-4600-8447-65de807d0eb6 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:00.529550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf21fdf-dceb-4775-baa4-8fc4a309fd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:00.530219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf21fdf-dceb-4775-baa4-8fc4a309fd38 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:00.530501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf21fdf-dceb-4775-baa4-8fc4a309fd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:00.530777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf21fdf-dceb-4775-baa4-8fc4a309fd38 None admin] PATH_INFO: `/v3/users/bb19e7f52e3f4b85bcc7952e23773462` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:00.531883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf21fdf-dceb-4775-baa4-8fc4a309fd38 None admin] RBAC: Authorizing `identity:get_user(user_id=bb19e7f52e3f4b85bcc7952e23773462)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:00.533335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf21fdf-dceb-4775-baa4-8fc4a309fd38 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:00.534951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 25/51] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:00 2022] GET /identity/v3/users/bb19e7f52e3f4b85bcc7952e23773462 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:00.540377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52a247-e955-470b-91f7-b25bc6705f75 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:00.583690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:00.639876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52a247-e955-470b-91f7-b25bc6705f75 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:00.674883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e488af-ecdc-4289-990e-ab410084d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:00.675541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e488af-ecdc-4289-990e-ab410084d99c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:00.675823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e488af-ecdc-4289-990e-ab410084d99c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:00.676101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e488af-ecdc-4289-990e-ab410084d99c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:00.677281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e488af-ecdc-4289-990e-ab410084d99c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:00.678817 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e488af-ecdc-4289-990e-ab410084d99c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:00.680500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 27/52] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:00.687520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf21fdf-dceb-4775-baa4-8fc4a309fd38 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:00.784545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d776b-36cf-4a3a-925f-c4654ebb247f None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:00.785057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d776b-36cf-4a3a-925f-c4654ebb247f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:00.785166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d776b-36cf-4a3a-925f-c4654ebb247f None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:00.785865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:00.786493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 28/53] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:29:00.788967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf21fdf-dceb-4775-baa4-8fc4a309fd38 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:00.791334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f654-c8a9-4bec-9a84-28fffd0691c7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:00.791628 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f654-c8a9-4bec-9a84-28fffd0691c7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:00.791913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f654-c8a9-4bec-9a84-28fffd0691c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:00.813802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0e93f654-c8a9-4bec-9a84-28fffd0691c7 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:00.815000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0e93f654-c8a9-4bec-9a84-28fffd0691c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:00.824056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d969fc3-bf2d-43e1-87d5-1f2e850b7ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:00.825529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d969fc3-bf2d-43e1-87d5-1f2e850b7ffd None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:00.825832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d969fc3-bf2d-43e1-87d5-1f2e850b7ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:00.826110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d969fc3-bf2d-43e1-87d5-1f2e850b7ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:00.826833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d969fc3-bf2d-43e1-87d5-1f2e850b7ffd None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.domain.id=default, user.id=bb19e7f52e3f4b85bcc7952e23773462) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:00.827140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d969fc3-bf2d-43e1-87d5-1f2e850b7ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:00.828601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d969fc3-bf2d-43e1-87d5-1f2e850b7ffd None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:00.834496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 26/54] 10.222.0.42 () {56 vars in 1322 bytes} [Sat Sep 17 03:29:00 2022] GET /identity/v3/role_assignments?user.id=bb19e7f52e3f4b85bcc7952e23773462&scope.domain.id=default&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:00.896015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 29/55] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:01.169372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d969fc3-bf2d-43e1-87d5-1f2e850b7ffd None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:01.270203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d969fc3-bf2d-43e1-87d5-1f2e850b7ffd None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:01.303409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf7204-a250-4b08-8097-7a6dcfe84ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:01.304027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf7204-a250-4b08-8097-7a6dcfe84ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:01.304305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf7204-a250-4b08-8097-7a6dcfe84ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:01.304575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf7204-a250-4b08-8097-7a6dcfe84ef7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:01.305665 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf7204-a250-4b08-8097-7a6dcfe84ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:01.307141 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf7204-a250-4b08-8097-7a6dcfe84ef7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:01.308870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 27/56] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:01.314728 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93f654-c8a9-4bec-9a84-28fffd0691c7 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:01.354537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:01.409068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93f654-c8a9-4bec-9a84-28fffd0691c7 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:01.441923 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5da468-99ce-47e1-aa97-c9d0c68e2e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:01.442457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5da468-99ce-47e1-aa97-c9d0c68e2e54 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:01.442735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5da468-99ce-47e1-aa97-c9d0c68e2e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:01.443007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5da468-99ce-47e1-aa97-c9d0c68e2e54 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:01.443646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5da468-99ce-47e1-aa97-c9d0c68e2e54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:01.445079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5da468-99ce-47e1-aa97-c9d0c68e2e54 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:01.565683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 30/57] 10.222.0.42 () {60 vars in 1140 bytes} [Sat Sep 17 03:29:01 2022] POST /identity/v3/projects => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:29:03.697784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537390-0746-4e8c-b8f0-0194c6fddbb2 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:03.698555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537390-0746-4e8c-b8f0-0194c6fddbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:03.698665 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537390-0746-4e8c-b8f0-0194c6fddbb2 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:03.700004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 28/58] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:03.705297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecdd02-4886-41f9-85e9-e8b38cbc6d9c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:03.705665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecdd02-4886-41f9-85e9-e8b38cbc6d9c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:03.705895 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecdd02-4886-41f9-85e9-e8b38cbc6d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:03.724246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-27ecdd02-4886-41f9-85e9-e8b38cbc6d9c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:03.725434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-27ecdd02-4886-41f9-85e9-e8b38cbc6d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:03.788855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 31/59] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:04.058353 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a537390-0746-4e8c-b8f0-0194c6fddbb2 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:04.089236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:04.157719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a537390-0746-4e8c-b8f0-0194c6fddbb2 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:04.190450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3b719-f377-4b5c-9d52-98422453c8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:04.191059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3b719-f377-4b5c-9d52-98422453c8fd None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:04.191335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3b719-f377-4b5c-9d52-98422453c8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:04.191603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3b719-f377-4b5c-9d52-98422453c8fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:04.192692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3b719-f377-4b5c-9d52-98422453c8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:04.194177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3b719-f377-4b5c-9d52-98422453c8fd None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:04.195700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 29/60] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:04.201458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecdd02-4886-41f9-85e9-e8b38cbc6d9c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:04.225024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:04.299833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecdd02-4886-41f9-85e9-e8b38cbc6d9c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:04.332716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149af54-2b5e-4efe-9d16-7d22c202f423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:04.333255 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149af54-2b5e-4efe-9d16-7d22c202f423 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:04.333537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149af54-2b5e-4efe-9d16-7d22c202f423 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:04.333843 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149af54-2b5e-4efe-9d16-7d22c202f423 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:04.334507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149af54-2b5e-4efe-9d16-7d22c202f423 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:04.335952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149af54-2b5e-4efe-9d16-7d22c202f423 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:04.432098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 32/61] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:29:04 2022] POST /identity/v3/users => generated 275 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:29:07.685321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf71e2a-bcd3-46da-b6cf-7fcf29d3b427 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:07.685890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf71e2a-bcd3-46da-b6cf-7fcf29d3b427 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:07.685890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf71e2a-bcd3-46da-b6cf-7fcf29d3b427 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:07.687267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 30/62] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:07.704343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb151d1e-62c1-444a-8671-9b587857b406 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:07.705342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb151d1e-62c1-444a-8671-9b587857b406 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:07.705848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb151d1e-62c1-444a-8671-9b587857b406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:07.735023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-bb151d1e-62c1-444a-8671-9b587857b406 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:07.736616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bb151d1e-62c1-444a-8671-9b587857b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:07.839044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 33/63] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:07.852382 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c375-5cce-4c4b-9d3f-945830ffe6de None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:07.852938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c375-5cce-4c4b-9d3f-945830ffe6de None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:07.853413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c375-5cce-4c4b-9d3f-945830ffe6de None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:07.854990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 31/64] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:07.860941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedea5e-b8b4-420b-aa6f-452d084fc910 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:07.861482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedea5e-b8b4-420b-aa6f-452d084fc910 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:07.861991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedea5e-b8b4-420b-aa6f-452d084fc910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:07.887406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5bedea5e-b8b4-420b-aa6f-452d084fc910 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:07.889115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5bedea5e-b8b4-420b-aa6f-452d084fc910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:07.891371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:07.983183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 34/65] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:08.168213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454f614-ef94-403e-b522-c77ed5817071 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.168882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454f614-ef94-403e-b522-c77ed5817071 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.169367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454f614-ef94-403e-b522-c77ed5817071 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.170982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 32/66] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:08.178159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedea5e-b8b4-420b-aa6f-452d084fc910 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:08.179685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf71510-9893-4b90-919f-b102839a31ff None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.180195 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf71510-9893-4b90-919f-b102839a31ff None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.180667 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf71510-9893-4b90-919f-b102839a31ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.210907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-2cf71510-9893-4b90-919f-b102839a31ff None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:08.212452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-2cf71510-9893-4b90-919f-b102839a31ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:08.290394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedea5e-b8b4-420b-aa6f-452d084fc910 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:08.329818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53de88-6420-4fad-8e6c-d5b39dc921ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:08.330490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53de88-6420-4fad-8e6c-d5b39dc921ef None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.330782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53de88-6420-4fad-8e6c-d5b39dc921ef None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.331066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53de88-6420-4fad-8e6c-d5b39dc921ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.339817 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:08.343146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 33/67] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:08.344429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf71510-9893-4b90-919f-b102839a31ff None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:08.344770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53de88-6420-4fad-8e6c-d5b39dc921ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ce53de88-6420-4fad-8e6c-d5b39dc921ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:08.350116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:08.354352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:08.355151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 35/68] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:08.359288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53de88-6420-4fad-8e6c-d5b39dc921ef None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:08.379813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:08.475758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53de88-6420-4fad-8e6c-d5b39dc921ef None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:08.484306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf71510-9893-4b90-919f-b102839a31ff None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:08.518616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff48b6-c222-4fc5-9f3b-d29b3cb228f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:08.519234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff48b6-c222-4fc5-9f3b-d29b3cb228f5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.519519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff48b6-c222-4fc5-9f3b-d29b3cb228f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.519614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582c00f-3c58-4c15-acd5-ed35315c7aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:08.519800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff48b6-c222-4fc5-9f3b-d29b3cb228f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.520236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582c00f-3c58-4c15-acd5-ed35315c7aaf None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.520469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff48b6-c222-4fc5-9f3b-d29b3cb228f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:08.520558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582c00f-3c58-4c15-acd5-ed35315c7aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.520778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582c00f-3c58-4c15-acd5-ed35315c7aaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.520866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff48b6-c222-4fc5-9f3b-d29b3cb228f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:08.522202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff48b6-c222-4fc5-9f3b-d29b3cb228f5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:08.531883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582c00f-3c58-4c15-acd5-ed35315c7aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:08.537092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 36/69] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:08.537092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0bc77-4dab-44e8-9f19-314e047316fe None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.537092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0bc77-4dab-44e8-9f19-314e047316fe None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.537092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0bc77-4dab-44e8-9f19-314e047316fe None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.537092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 37/70] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:08.537092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f15e30-d374-4c2a-b701-1d242d745b39 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.537092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f15e30-d374-4c2a-b701-1d242d745b39 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.537546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f15e30-d374-4c2a-b701-1d242d745b39 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.538649 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 38/71] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d582c00f-3c58-4c15-acd5-ed35315c7aaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:08.540868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:08.541692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:08.543075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 34/72] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:08.544495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a365c9-6ee2-4172-8e59-7b9fbb2b1686 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.544792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a365c9-6ee2-4172-8e59-7b9fbb2b1686 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.545069 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a365c9-6ee2-4172-8e59-7b9fbb2b1686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.546630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582c00f-3c58-4c15-acd5-ed35315c7aaf None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:08.563423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-68a365c9-6ee2-4172-8e59-7b9fbb2b1686 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:08.564620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-68a365c9-6ee2-4172-8e59-7b9fbb2b1686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:08.620702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:08.641751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 39/73] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:08.641751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bee4c-da0a-4d72-ae5a-e3b5a82adf59 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.641751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bee4c-da0a-4d72-ae5a-e3b5a82adf59 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.641751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bee4c-da0a-4d72-ae5a-e3b5a82adf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.653524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d04bee4c-da0a-4d72-ae5a-e3b5a82adf59 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:08.654929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d04bee4c-da0a-4d72-ae5a-e3b5a82adf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:08.659544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:08.674103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582c00f-3c58-4c15-acd5-ed35315c7aaf None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:08.711025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff105d-367f-4d7b-8d33-f531dd52fe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:08.711936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff105d-367f-4d7b-8d33-f531dd52fe11 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.712222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff105d-367f-4d7b-8d33-f531dd52fe11 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.712492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff105d-367f-4d7b-8d33-f531dd52fe11 None admin] PATH_INFO: `/v3/users/c3e17b4e2c274478a5af0c474f6198f3` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.724619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 40/74] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:08.726001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bee4c-da0a-4d72-ae5a-e3b5a82adf59 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:08.735607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff105d-367f-4d7b-8d33-f531dd52fe11 None admin] RBAC: Authorizing `identity:get_user(user_id=c3e17b4e2c274478a5af0c474f6198f3)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:08.737905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff105d-367f-4d7b-8d33-f531dd52fe11 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:08.740503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 35/75] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:08 2022] GET /identity/v3/users/c3e17b4e2c274478a5af0c474f6198f3 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:08.741462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff105d-367f-4d7b-8d33-f531dd52fe11 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:08.849621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bee4c-da0a-4d72-ae5a-e3b5a82adf59 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:08.865036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff105d-367f-4d7b-8d33-f531dd52fe11 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:08.876912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:08.897205 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fd05d-c47f-437f-b18d-24539aae3670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:08.898329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fd05d-c47f-437f-b18d-24539aae3670 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.898823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fd05d-c47f-437f-b18d-24539aae3670 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.899276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fd05d-c47f-437f-b18d-24539aae3670 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.900123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fd05d-c47f-437f-b18d-24539aae3670 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:08.900549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fd05d-c47f-437f-b18d-24539aae3670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:08.902899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fd05d-c47f-437f-b18d-24539aae3670 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:08.904640 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bae16-45cf-4890-9251-f8454a64e507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:08.905481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bae16-45cf-4890-9251-f8454a64e507 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:08.906582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bae16-45cf-4890-9251-f8454a64e507 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:08.907040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bae16-45cf-4890-9251-f8454a64e507 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:08.913448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 41/76] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:08.914239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fd05d-c47f-437f-b18d-24539aae3670 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:08.921128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bae16-45cf-4890-9251-f8454a64e507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-531bae16-45cf-4890-9251-f8454a64e507 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:08.926652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:08.927588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:08.928676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:08.930615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 36/77] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:08.936586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bae16-45cf-4890-9251-f8454a64e507 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:08.948918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:09.019704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fd05d-c47f-437f-b18d-24539aae3670 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:09.041353 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bae16-45cf-4890-9251-f8454a64e507 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:09.052941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c3a7b-5503-49bb-bfdc-573bed04840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:09.053480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c3a7b-5503-49bb-bfdc-573bed04840e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:09.053786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c3a7b-5503-49bb-bfdc-573bed04840e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:09.054085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c3a7b-5503-49bb-bfdc-573bed04840e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:09.076264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb4d18-e315-4e99-bc54-b4c25c7e3bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:09.076884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4d18-e315-4e99-bc54-b4c25c7e3bfd None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:09.077162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4d18-e315-4e99-bc54-b4c25c7e3bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:09.077433 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4d18-e315-4e99-bc54-b4c25c7e3bfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:09.078577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb4d18-e315-4e99-bc54-b4c25c7e3bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:09.080028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb4d18-e315-4e99-bc54-b4c25c7e3bfd None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:09.081661 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 37/78] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:09.082441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb4d18-e315-4e99-bc54-b4c25c7e3bfd None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:09.103979 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:09.123745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c3a7b-5503-49bb-bfdc-573bed04840e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:09.125204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c3a7b-5503-49bb-bfdc-573bed04840e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:09.157014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:09.179483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb4d18-e315-4e99-bc54-b4c25c7e3bfd None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:09.213073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799e3b7-0c04-4338-ab0a-603fe945b512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:09.213693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799e3b7-0c04-4338-ab0a-603fe945b512 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:09.213998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799e3b7-0c04-4338-ab0a-603fe945b512 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:09.214273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799e3b7-0c04-4338-ab0a-603fe945b512 None admin] PATH_INFO: `/v3/users/bb19e7f52e3f4b85bcc7952e23773462` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:09.216027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799e3b7-0c04-4338-ab0a-603fe945b512 None admin] RBAC: Authorizing `identity:get_user(user_id=bb19e7f52e3f4b85bcc7952e23773462)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:09.217466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799e3b7-0c04-4338-ab0a-603fe945b512 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:09.219060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 38/79] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:09 2022] GET /identity/v3/users/bb19e7f52e3f4b85bcc7952e23773462 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:09.219829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799e3b7-0c04-4338-ab0a-603fe945b512 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:09.223754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 42/80] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:09.226487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c3a7b-5503-49bb-bfdc-573bed04840e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:09.308241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:09.322395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799e3b7-0c04-4338-ab0a-603fe945b512 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:09.335908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c3a7b-5503-49bb-bfdc-573bed04840e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:09.358897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a0f73-d497-4673-a9e7-9a123d4c72c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:09.359506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a0f73-d497-4673-a9e7-9a123d4c72c5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:09.359790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a0f73-d497-4673-a9e7-9a123d4c72c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:09.360073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a0f73-d497-4673-a9e7-9a123d4c72c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:09.360738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a0f73-d497-4673-a9e7-9a123d4c72c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:09.361047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a0f73-d497-4673-a9e7-9a123d4c72c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:09.362479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a0f73-d497-4673-a9e7-9a123d4c72c5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:09.370563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 39/81] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:09.371269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a0f73-d497-4673-a9e7-9a123d4c72c5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:09.379802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:09.381656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0a400-1ada-49cc-96e2-1d378b510fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:09.382395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0a400-1ada-49cc-96e2-1d378b510fff None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:09.382675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0a400-1ada-49cc-96e2-1d378b510fff None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:09.382950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0a400-1ada-49cc-96e2-1d378b510fff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:09.393899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0a400-1ada-49cc-96e2-1d378b510fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a6b0a400-1ada-49cc-96e2-1d378b510fff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:09.398746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:09.399814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 43/82] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:09.400723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0a400-1ada-49cc-96e2-1d378b510fff None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:09.475582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a0f73-d497-4673-a9e7-9a123d4c72c5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:09.501625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0a400-1ada-49cc-96e2-1d378b510fff None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:09.508582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b730d82-84c9-437f-aa45-f6ef979650eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:09.509191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b730d82-84c9-437f-aa45-f6ef979650eb None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:09.509480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b730d82-84c9-437f-aa45-f6ef979650eb None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:09.509781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b730d82-84c9-437f-aa45-f6ef979650eb None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:09.510437 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b730d82-84c9-437f-aa45-f6ef979650eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:09.512013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b730d82-84c9-437f-aa45-f6ef979650eb None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:09.539802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad42fa-c355-40bd-b2b7-e5dd68f84760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:09.542084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad42fa-c355-40bd-b2b7-e5dd68f84760 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:09.542084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad42fa-c355-40bd-b2b7-e5dd68f84760 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:09.542084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad42fa-c355-40bd-b2b7-e5dd68f84760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:09.555142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:09.609611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad42fa-c355-40bd-b2b7-e5dd68f84760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:09.611067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad42fa-c355-40bd-b2b7-e5dd68f84760 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:09.616204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 40/83] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:29:09 2022] POST /identity/v3/users => generated 289 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:29:09.617119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b730d82-84c9-437f-aa45-f6ef979650eb None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:09.706003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:09.733955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 44/84] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:09.733955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad42fa-c355-40bd-b2b7-e5dd68f84760 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:09.764164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b730d82-84c9-437f-aa45-f6ef979650eb None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:09.781332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:09.812281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310d5cb-d65e-4ad2-a067-b55ab5e3cd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:09.812923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310d5cb-d65e-4ad2-a067-b55ab5e3cd8b None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:09.813199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310d5cb-d65e-4ad2-a067-b55ab5e3cd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:09.813471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310d5cb-d65e-4ad2-a067-b55ab5e3cd8b None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:09.814656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310d5cb-d65e-4ad2-a067-b55ab5e3cd8b None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:09.816130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310d5cb-d65e-4ad2-a067-b55ab5e3cd8b None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:09.817843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 41/85] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:09 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:09.818616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310d5cb-d65e-4ad2-a067-b55ab5e3cd8b None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:09.853851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad42fa-c355-40bd-b2b7-e5dd68f84760 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:09.895787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb7929-930b-4191-a5e8-072354586ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:09.896347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb7929-930b-4191-a5e8-072354586ffc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:09.896640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb7929-930b-4191-a5e8-072354586ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:09.896919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb7929-930b-4191-a5e8-072354586ffc None admin] PATH_INFO: `/v3/users/c3e17b4e2c274478a5af0c474f6198f3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:09.898775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb7929-930b-4191-a5e8-072354586ffc None admin] RBAC: Authorizing `identity:get_user(user_id=c3e17b4e2c274478a5af0c474f6198f3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:09.900245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb7929-930b-4191-a5e8-072354586ffc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:09.902253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 45/86] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:09 2022] GET /identity/v3/users/c3e17b4e2c274478a5af0c474f6198f3 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:09.903106 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb7929-930b-4191-a5e8-072354586ffc None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:09.950891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310d5cb-d65e-4ad2-a067-b55ab5e3cd8b None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:09.982446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:09.989617 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177fa37-9d2c-452b-9c23-266659d52f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:09.990670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177fa37-9d2c-452b-9c23-266659d52f15 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:09.991173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177fa37-9d2c-452b-9c23-266659d52f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:09.991666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177fa37-9d2c-452b-9c23-266659d52f15 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:09.992849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177fa37-9d2c-452b-9c23-266659d52f15 None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:09.993485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177fa37-9d2c-452b-9c23-266659d52f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:09.995240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177fa37-9d2c-452b-9c23-266659d52f15 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:10.007997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 42/87] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:10.008848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177fa37-9d2c-452b-9c23-266659d52f15 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:10.037914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb7929-930b-4191-a5e8-072354586ffc None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:10.050418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:10.075418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128221f5-4a90-4bf4-80ee-1cb367966673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:10.076055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128221f5-4a90-4bf4-80ee-1cb367966673 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:10.076338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128221f5-4a90-4bf4-80ee-1cb367966673 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:10.076613 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128221f5-4a90-4bf4-80ee-1cb367966673 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:10.077873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128221f5-4a90-4bf4-80ee-1cb367966673 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:10.079440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128221f5-4a90-4bf4-80ee-1cb367966673 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:10.081080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 46/88] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:09 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:10.085861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128221f5-4a90-4bf4-80ee-1cb367966673 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:10.135447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177fa37-9d2c-452b-9c23-266659d52f15 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:10.173847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe301b4-9ee3-4500-ba87-6db744fc3f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:10.174702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe301b4-9ee3-4500-ba87-6db744fc3f79 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:10.175145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe301b4-9ee3-4500-ba87-6db744fc3f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:10.175556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe301b4-9ee3-4500-ba87-6db744fc3f79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:10.176416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe301b4-9ee3-4500-ba87-6db744fc3f79 None admin] RBAC: Adding query filter params (role.id=d801ebea910445a7a6100b35e07630dd, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6, user.id=c3e17b4e2c274478a5af0c474f6198f3) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:10.176854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe301b4-9ee3-4500-ba87-6db744fc3f79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:10.178455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe301b4-9ee3-4500-ba87-6db744fc3f79 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:10.185096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 43/89] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:10 2022] GET /identity/v3/role_assignments?user.id=c3e17b4e2c274478a5af0c474f6198f3&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=d801ebea910445a7a6100b35e07630dd => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:10.186236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe301b4-9ee3-4500-ba87-6db744fc3f79 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:10.209421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128221f5-4a90-4bf4-80ee-1cb367966673 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:10.248666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965c94e-6ff8-4e5c-a7dc-625ec0df85b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:10.249480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965c94e-6ff8-4e5c-a7dc-625ec0df85b2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:10.249921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965c94e-6ff8-4e5c-a7dc-625ec0df85b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:10.250288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965c94e-6ff8-4e5c-a7dc-625ec0df85b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:10.252891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:10.304312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:10.305257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe301b4-9ee3-4500-ba87-6db744fc3f79 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:10.328763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965c94e-6ff8-4e5c-a7dc-625ec0df85b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:10.330291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965c94e-6ff8-4e5c-a7dc-625ec0df85b2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:10.344733 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf187f1d-b200-4699-9d60-f8ec484c0489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:10.345770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf187f1d-b200-4699-9d60-f8ec484c0489 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:10.346255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf187f1d-b200-4699-9d60-f8ec484c0489 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:10.346737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf187f1d-b200-4699-9d60-f8ec484c0489 None admin] PATH_INFO: `/v3/users/c3e17b4e2c274478a5af0c474f6198f3` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:10.348184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf187f1d-b200-4699-9d60-f8ec484c0489 None admin] RBAC: Authorizing `identity:get_user(user_id=c3e17b4e2c274478a5af0c474f6198f3)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:10.349981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf187f1d-b200-4699-9d60-f8ec484c0489 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:10.351848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 44/90] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:10 2022] GET /identity/v3/users/c3e17b4e2c274478a5af0c474f6198f3 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:10.352834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf187f1d-b200-4699-9d60-f8ec484c0489 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:10.502382 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf187f1d-b200-4699-9d60-f8ec484c0489 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:10.547402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:10.549709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 47/91] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:10.550553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965c94e-6ff8-4e5c-a7dc-625ec0df85b2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:10.571340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef32d8-8503-4cf3-9c61-850f021fd8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:10.572008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef32d8-8503-4cf3-9c61-850f021fd8ae None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:10.572287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef32d8-8503-4cf3-9c61-850f021fd8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:10.572558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef32d8-8503-4cf3-9c61-850f021fd8ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:10.573600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef32d8-8503-4cf3-9c61-850f021fd8ae None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6, user.id=bb19e7f52e3f4b85bcc7952e23773462) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:10.573929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef32d8-8503-4cf3-9c61-850f021fd8ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:10.575427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef32d8-8503-4cf3-9c61-850f021fd8ae None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:10.584548 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 45/92] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:10 2022] GET /identity/v3/role_assignments?user.id=bb19e7f52e3f4b85bcc7952e23773462&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:10.587514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef32d8-8503-4cf3-9c61-850f021fd8ae None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:10.708778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:10.756670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965c94e-6ff8-4e5c-a7dc-625ec0df85b2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:10.771578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef32d8-8503-4cf3-9c61-850f021fd8ae None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:10.830147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6d341-928c-4e6f-8103-9a5842b86fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:10.831583 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6d341-928c-4e6f-8103-9a5842b86fce None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:10.832386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30262e1-9ce1-4e6c-be82-1a4622776029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:10.832831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6d341-928c-4e6f-8103-9a5842b86fce None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:10.833669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30262e1-9ce1-4e6c-be82-1a4622776029 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:10.834109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6d341-928c-4e6f-8103-9a5842b86fce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:10.835164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30262e1-9ce1-4e6c-be82-1a4622776029 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:10.835584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30262e1-9ce1-4e6c-be82-1a4622776029 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:10.839207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30262e1-9ce1-4e6c-be82-1a4622776029 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:10.840997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30262e1-9ce1-4e6c-be82-1a4622776029 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:10.844149 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 46/93] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:10 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:10.854509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30262e1-9ce1-4e6c-be82-1a4622776029 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:10.947226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:10.989518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6d341-928c-4e6f-8103-9a5842b86fce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:10.991514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6d341-928c-4e6f-8103-9a5842b86fce None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:11.072321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30262e1-9ce1-4e6c-be82-1a4622776029 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:11.154155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:11.156867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b462-ba94-4860-a324-9f80156b880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:11.157493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b462-ba94-4860-a324-9f80156b880c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:11.157799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b462-ba94-4860-a324-9f80156b880c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:11.158086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b462-ba94-4860-a324-9f80156b880c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:11.158830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146b462-ba94-4860-a324-9f80156b880c None admin] RBAC: Adding query filter params (role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6, user.id=c3e17b4e2c274478a5af0c474f6198f3) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:11.159147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146b462-ba94-4860-a324-9f80156b880c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:11.160624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146b462-ba94-4860-a324-9f80156b880c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:11.181308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 47/94] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:10 2022] GET /identity/v3/role_assignments?user.id=c3e17b4e2c274478a5af0c474f6198f3&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:11.186224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 48/95] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:11.199228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b462-ba94-4860-a324-9f80156b880c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:11.363367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b462-ba94-4860-a324-9f80156b880c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:11.393642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:11.410688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191134f-a3be-4d26-9a4a-741d900e2100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:11.413666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191134f-a3be-4d26-9a4a-741d900e2100 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:11.413666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191134f-a3be-4d26-9a4a-741d900e2100 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:11.413666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191134f-a3be-4d26-9a4a-741d900e2100 None admin] PATH_INFO: `/v3/projects/58bf3c47d4214522bc44853d51d222db` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:11.413666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191134f-a3be-4d26-9a4a-741d900e2100 None admin] RBAC: Authorizing `identity:get_project(project_id=58bf3c47d4214522bc44853d51d222db)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:11.414741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191134f-a3be-4d26-9a4a-741d900e2100 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:11.416824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 48/96] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:11 2022] GET /identity/v3/projects/58bf3c47d4214522bc44853d51d222db => generated 312 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:11.423244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6d341-928c-4e6f-8103-9a5842b86fce None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:11.558450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6d341-928c-4e6f-8103-9a5842b86fce None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:11.597625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ca18a-10ca-4610-81b8-a7332a156874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:11.598273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ca18a-10ca-4610-81b8-a7332a156874 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:11.598554 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ca18a-10ca-4610-81b8-a7332a156874 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:11.598830 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ca18a-10ca-4610-81b8-a7332a156874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:11.599565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ca18a-10ca-4610-81b8-a7332a156874 None admin] RBAC: Adding query filter params (role.id=d801ebea910445a7a6100b35e07630dd, scope.project.id=58bf3c47d4214522bc44853d51d222db, user.id=c3e17b4e2c274478a5af0c474f6198f3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:11.599876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ca18a-10ca-4610-81b8-a7332a156874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:11.601305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ca18a-10ca-4610-81b8-a7332a156874 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:11.610138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 49/97] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:11 2022] GET /identity/v3/role_assignments?user.id=c3e17b4e2c274478a5af0c474f6198f3&scope.project.id=58bf3c47d4214522bc44853d51d222db&role.id=d801ebea910445a7a6100b35e07630dd => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:12.613112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc02ddb-9169-44e1-aedb-4f50cbcaf723 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:12.614165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc02ddb-9169-44e1-aedb-4f50cbcaf723 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:12.614721 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc02ddb-9169-44e1-aedb-4f50cbcaf723 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:12.616534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 49/98] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:12.623353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea814033-9bb4-4933-a470-679e276c891e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:12.624305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea814033-9bb4-4933-a470-679e276c891e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:12.624787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea814033-9bb4-4933-a470-679e276c891e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:12.644742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ea814033-9bb4-4933-a470-679e276c891e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:12.646343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ea814033-9bb4-4933-a470-679e276c891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:12.702955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:12.737092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 50/99] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:12.774610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cba8b-d870-446d-b2f4-011557349100 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:12.774894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cba8b-d870-446d-b2f4-011557349100 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:12.775174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cba8b-d870-446d-b2f4-011557349100 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:12.776529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 50/100] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:12.781977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d10fe-8818-4a75-a23e-7ce8797eb293 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:12.782271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d10fe-8818-4a75-a23e-7ce8797eb293 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:12.782552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d10fe-8818-4a75-a23e-7ce8797eb293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:12.801116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0e8d10fe-8818-4a75-a23e-7ce8797eb293 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:12.802108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0e8d10fe-8818-4a75-a23e-7ce8797eb293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:12.887892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 51/101] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:13.146706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cba8b-d870-446d-b2f4-011557349100 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:13.275056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cba8b-d870-446d-b2f4-011557349100 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:13.315897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c82ff-e445-4616-b5b3-ac826175612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:13.316538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c82ff-e445-4616-b5b3-ac826175612f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:13.316823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c82ff-e445-4616-b5b3-ac826175612f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:13.317099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c82ff-e445-4616-b5b3-ac826175612f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:13.320094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c82ff-e445-4616-b5b3-ac826175612f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:13.321640 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c82ff-e445-4616-b5b3-ac826175612f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:13.323788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 51/102] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:13.329457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d10fe-8818-4a75-a23e-7ce8797eb293 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:13.446256 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c82ff-e445-4616-b5b3-ac826175612f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:13.476193 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d10fe-8818-4a75-a23e-7ce8797eb293 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:13.478004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:13.534462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe964a3-65a7-4e73-a8bf-858e688e40bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:13.535086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe964a3-65a7-4e73-a8bf-858e688e40bd None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:13.535373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe964a3-65a7-4e73-a8bf-858e688e40bd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:13.535651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe964a3-65a7-4e73-a8bf-858e688e40bd None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:13.536306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe964a3-65a7-4e73-a8bf-858e688e40bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:13.537770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe964a3-65a7-4e73-a8bf-858e688e40bd None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:13.589978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:13.639411 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 52/103] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:29:13 2022] POST /identity/v3/projects => generated 302 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:29:13.640748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87903d3-7b76-4948-97f7-33adb3286dcb None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:13.641045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87903d3-7b76-4948-97f7-33adb3286dcb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:13.641324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87903d3-7b76-4948-97f7-33adb3286dcb None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:13.642599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 53/104] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:13.653522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c83e2-f482-4f7d-a472-9fae6deaa099 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:13.654646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c83e2-f482-4f7d-a472-9fae6deaa099 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:13.654946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c83e2-f482-4f7d-a472-9fae6deaa099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:13.658192 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c82ff-e445-4616-b5b3-ac826175612f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:13.699442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-455c83e2-f482-4f7d-a472-9fae6deaa099 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:13.700754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-455c83e2-f482-4f7d-a472-9fae6deaa099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:13.756277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1e319-2d8a-42ff-a08d-fbb11b6909f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:13.756980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1e319-2d8a-42ff-a08d-fbb11b6909f5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:13.757280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1e319-2d8a-42ff-a08d-fbb11b6909f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:13.757565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1e319-2d8a-42ff-a08d-fbb11b6909f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:13.784057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1e319-2d8a-42ff-a08d-fbb11b6909f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-29a1e319-2d8a-42ff-a08d-fbb11b6909f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:13.789705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:13.790634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:13.791435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 52/105] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:13.820752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 54/106] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:13.822479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1e319-2d8a-42ff-a08d-fbb11b6909f5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:13.946324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a910980-693a-451f-bcfc-286e6fb874dd None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:13.946701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a910980-693a-451f-bcfc-286e6fb874dd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:13.946885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a910980-693a-451f-bcfc-286e6fb874dd None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:13.948642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 55/107] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:29:13.959079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4948916-6fa9-4fcf-a4f3-8a3fd93aaec2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:13.959688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4948916-6fa9-4fcf-a4f3-8a3fd93aaec2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:13.960214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4948916-6fa9-4fcf-a4f3-8a3fd93aaec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:13.971026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1e319-2d8a-42ff-a08d-fbb11b6909f5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:13.987497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d4948916-6fa9-4fcf-a4f3-8a3fd93aaec2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:13.989436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d4948916-6fa9-4fcf-a4f3-8a3fd93aaec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:13.998285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:14.026394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285b4d-6732-4b21-b3aa-43efb5cef7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:14.027026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285b4d-6732-4b21-b3aa-43efb5cef7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:14.027465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285b4d-6732-4b21-b3aa-43efb5cef7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:14.027746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285b4d-6732-4b21-b3aa-43efb5cef7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:14.028421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d285b4d-6732-4b21-b3aa-43efb5cef7d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:14.028725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d285b4d-6732-4b21-b3aa-43efb5cef7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:14.030354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d285b4d-6732-4b21-b3aa-43efb5cef7d9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:14.039479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 53/108] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:14.046323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285b4d-6732-4b21-b3aa-43efb5cef7d9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:14.092729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 56/109] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:14.202493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285b4d-6732-4b21-b3aa-43efb5cef7d9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:14.238087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4948916-6fa9-4fcf-a4f3-8a3fd93aaec2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:14.245274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705121a-53f7-42a6-9cda-b62283877dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:14.246551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705121a-53f7-42a6-9cda-b62283877dca None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:14.246876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705121a-53f7-42a6-9cda-b62283877dca None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:14.247155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705121a-53f7-42a6-9cda-b62283877dca None admin] PATH_INFO: `/v3/users/efb5876c14074177a961c3a7fd2032c5` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:14.272756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:14.275055 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705121a-53f7-42a6-9cda-b62283877dca None admin] RBAC: Authorizing `identity:get_user(user_id=efb5876c14074177a961c3a7fd2032c5)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:14.276552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705121a-53f7-42a6-9cda-b62283877dca None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:14.279187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 54/110] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:14 2022] GET /identity/v3/users/efb5876c14074177a961c3a7fd2032c5 => generated 289 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:14.285105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705121a-53f7-42a6-9cda-b62283877dca None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:14.323691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:14.437186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705121a-53f7-42a6-9cda-b62283877dca None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:14.437655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4948916-6fa9-4fcf-a4f3-8a3fd93aaec2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:14.476965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e858d5c-0033-41f6-bb39-22a0c3ee2be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:14.477650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858d5c-0033-41f6-bb39-22a0c3ee2be8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:14.477962 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858d5c-0033-41f6-bb39-22a0c3ee2be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:14.478237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858d5c-0033-41f6-bb39-22a0c3ee2be8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:14.491294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e858d5c-0033-41f6-bb39-22a0c3ee2be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-7e858d5c-0033-41f6-bb39-22a0c3ee2be8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:14.497508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:14.502731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:14.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 57/111] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:14.504174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96f324-49a5-4349-8ec4-c020487de3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:14.504174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96f324-49a5-4349-8ec4-c020487de3ca None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:14.504174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96f324-49a5-4349-8ec4-c020487de3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:14.504174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96f324-49a5-4349-8ec4-c020487de3ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:14.507044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e858d5c-0033-41f6-bb39-22a0c3ee2be8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:14.638246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:14.642582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96f324-49a5-4349-8ec4-c020487de3ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:14.644107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96f324-49a5-4349-8ec4-c020487de3ca None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:14.673480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e858d5c-0033-41f6-bb39-22a0c3ee2be8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:14.723950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:14.751195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb2977-7e99-4a5e-a8ca-1ad785e5063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:14.751739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb2977-7e99-4a5e-a8ca-1ad785e5063a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:14.752085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb2977-7e99-4a5e-a8ca-1ad785e5063a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:14.752366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb2977-7e99-4a5e-a8ca-1ad785e5063a None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:14.753042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb2977-7e99-4a5e-a8ca-1ad785e5063a None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:14.753393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb2977-7e99-4a5e-a8ca-1ad785e5063a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:14.754814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb2977-7e99-4a5e-a8ca-1ad785e5063a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:14.771810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 58/112] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:14.772650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb2977-7e99-4a5e-a8ca-1ad785e5063a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:14.812906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 55/113] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 03:29:14.813666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96f324-49a5-4349-8ec4-c020487de3ca None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:14.912928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb2977-7e99-4a5e-a8ca-1ad785e5063a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:14.921772 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96f324-49a5-4349-8ec4-c020487de3ca None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:14.949856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87a71f-6407-40fa-a1d6-f39019d8b908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:14.950636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87a71f-6407-40fa-a1d6-f39019d8b908 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:14.950976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87a71f-6407-40fa-a1d6-f39019d8b908 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:14.951255 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87a71f-6407-40fa-a1d6-f39019d8b908 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:14.958141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:14.959390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbee01-8691-4099-bfb7-4b025ab89ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:14.960499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbee01-8691-4099-bfb7-4b025ab89ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:14.961014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbee01-8691-4099-bfb7-4b025ab89ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:14.961524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbee01-8691-4099-bfb7-4b025ab89ca8 None admin] PATH_INFO: `/v3/users/c3e17b4e2c274478a5af0c474f6198f3` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:14.963573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbee01-8691-4099-bfb7-4b025ab89ca8 None admin] RBAC: Authorizing `identity:get_user(user_id=c3e17b4e2c274478a5af0c474f6198f3)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:14.966697 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87a71f-6407-40fa-a1d6-f39019d8b908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:14.966842 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbee01-8691-4099-bfb7-4b025ab89ca8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:14.968976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 56/114] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:14 2022] GET /identity/v3/users/c3e17b4e2c274478a5af0c474f6198f3 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:14.969591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbee01-8691-4099-bfb7-4b025ab89ca8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ab87a71f-6407-40fa-a1d6-f39019d8b908 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:14.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:14.977323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 59/115] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:29:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:14.978213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98783951-8b50-4b43-a141-5021bcc0dc4c None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:14.978933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98783951-8b50-4b43-a141-5021bcc0dc4c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:14.979584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98783951-8b50-4b43-a141-5021bcc0dc4c None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:14.982641 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:14.984076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 60/116] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:14.989846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98783951-8b50-4b43-a141-5021bcc0dc4c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:15.088687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbee01-8691-4099-bfb7-4b025ab89ca8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:15.109224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98783951-8b50-4b43-a141-5021bcc0dc4c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:15.126018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8041434-75d3-4161-9802-fdff8e548ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:15.126650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8041434-75d3-4161-9802-fdff8e548ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.126935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8041434-75d3-4161-9802-fdff8e548ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.127211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8041434-75d3-4161-9802-fdff8e548ec0 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.128335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8041434-75d3-4161-9802-fdff8e548ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:15.129815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8041434-75d3-4161-9802-fdff8e548ec0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:15.131688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 57/117] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:14 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:15.132334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8041434-75d3-4161-9802-fdff8e548ec0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:15.150273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ea790-806b-4adf-a59c-f51a55ac9767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:15.151105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ea790-806b-4adf-a59c-f51a55ac9767 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.151641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ea790-806b-4adf-a59c-f51a55ac9767 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.152233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ea790-806b-4adf-a59c-f51a55ac9767 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.153425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ea790-806b-4adf-a59c-f51a55ac9767 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:15.155095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ea790-806b-4adf-a59c-f51a55ac9767 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:15.156881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 61/118] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:14 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:15.158176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d0a3d-832e-4e3f-a905-005b70a3457d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.158465 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d0a3d-832e-4e3f-a905-005b70a3457d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.158749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d0a3d-832e-4e3f-a905-005b70a3457d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.178339 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d84d0a3d-832e-4e3f-a905-005b70a3457d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:15.179778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d84d0a3d-832e-4e3f-a905-005b70a3457d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:15.208541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:15.239952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8041434-75d3-4161-9802-fdff8e548ec0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:15.240760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:15.251813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 62/119] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:15.258122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fb028-974d-4995-9bdb-376908f6242e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.258122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fb028-974d-4995-9bdb-376908f6242e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.258122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fb028-974d-4995-9bdb-376908f6242e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.258122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 63/120] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:15.259616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fb028-974d-4995-9bdb-376908f6242e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:15.279982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce635e8-7379-4d61-96f4-a25c9eb766e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:15.280919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce635e8-7379-4d61-96f4-a25c9eb766e7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.281450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce635e8-7379-4d61-96f4-a25c9eb766e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.281959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce635e8-7379-4d61-96f4-a25c9eb766e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.282960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce635e8-7379-4d61-96f4-a25c9eb766e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:15.283443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce635e8-7379-4d61-96f4-a25c9eb766e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:15.285124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce635e8-7379-4d61-96f4-a25c9eb766e7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:15.294380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 58/121] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:15.295391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce635e8-7379-4d61-96f4-a25c9eb766e7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:15.396899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fb028-974d-4995-9bdb-376908f6242e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:15.408724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce635e8-7379-4d61-96f4-a25c9eb766e7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:15.438083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74280502-84e5-4f3d-aa6e-5178302b0b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:15.438712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74280502-84e5-4f3d-aa6e-5178302b0b14 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.439221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74280502-84e5-4f3d-aa6e-5178302b0b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.439348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74280502-84e5-4f3d-aa6e-5178302b0b14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.440084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74280502-84e5-4f3d-aa6e-5178302b0b14 None admin] RBAC: Adding query filter params (role.id=ed414442e2d24f2f99e36d88ade1835e, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6, user.id=efb5876c14074177a961c3a7fd2032c5) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:15.440389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74280502-84e5-4f3d-aa6e-5178302b0b14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:15.441897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74280502-84e5-4f3d-aa6e-5178302b0b14 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:15.447682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 64/122] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:15 2022] GET /identity/v3/role_assignments?user.id=efb5876c14074177a961c3a7fd2032c5&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=ed414442e2d24f2f99e36d88ade1835e => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:15.449340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eee29c-0287-440e-95cd-0fc16cc02d4f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.449572 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eee29c-0287-440e-95cd-0fc16cc02d4f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.449868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eee29c-0287-440e-95cd-0fc16cc02d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.452711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a52d29-74e3-45a6-ae72-5368145e71c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:15.453649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a52d29-74e3-45a6-ae72-5368145e71c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.454171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a52d29-74e3-45a6-ae72-5368145e71c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.454623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a52d29-74e3-45a6-ae72-5368145e71c0 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/users/c3e17b4e2c274478a5af0c474f6198f3/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.457200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a52d29-74e3-45a6-ae72-5368145e71c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6, user_id=c3e17b4e2c274478a5af0c474f6198f3, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:15.459085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a52d29-74e3-45a6-ae72-5368145e71c0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:15.462070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:15.474524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-91eee29c-0287-440e-95cd-0fc16cc02d4f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:15.475742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-91eee29c-0287-440e-95cd-0fc16cc02d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:15.535537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:15.540276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 59/123] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:29:15 2022] PUT /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/users/c3e17b4e2c274478a5af0c474f6198f3/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:15.541544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a52d29-74e3-45a6-ae72-5368145e71c0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:15.566204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 65/124] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:15.672396 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eee29c-0287-440e-95cd-0fc16cc02d4f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:15.715233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a52d29-74e3-45a6-ae72-5368145e71c0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:15.764033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8cf12-2c5e-44f2-a3d2-2b53bae1002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:15.766121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8cf12-2c5e-44f2-a3d2-2b53bae1002a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.766594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8cf12-2c5e-44f2-a3d2-2b53bae1002a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.767041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8cf12-2c5e-44f2-a3d2-2b53bae1002a None admin] PATH_INFO: `/v3/users/bb19e7f52e3f4b85bcc7952e23773462` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.770064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b8cf12-2c5e-44f2-a3d2-2b53bae1002a None admin] RBAC: Authorizing `identity:get_user(user_id=bb19e7f52e3f4b85bcc7952e23773462)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:15.771943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b8cf12-2c5e-44f2-a3d2-2b53bae1002a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:15.774533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 60/125] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:15 2022] GET /identity/v3/users/bb19e7f52e3f4b85bcc7952e23773462 => generated 247 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:15.782713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8cf12-2c5e-44f2-a3d2-2b53bae1002a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:15.815515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eee29c-0287-440e-95cd-0fc16cc02d4f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:15.853656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac1b2c-b985-437e-aa2d-e21904da6bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:15.857898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac1b2c-b985-437e-aa2d-e21904da6bee None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.857898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac1b2c-b985-437e-aa2d-e21904da6bee None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.857898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac1b2c-b985-437e-aa2d-e21904da6bee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.866657 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac1b2c-b985-437e-aa2d-e21904da6bee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-cfac1b2c-b985-437e-aa2d-e21904da6bee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:15.877767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:15.878658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 66/126] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:15.879415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac1b2c-b985-437e-aa2d-e21904da6bee None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:15.882198 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:15.921754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8cf12-2c5e-44f2-a3d2-2b53bae1002a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:15.938267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:15.978436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042293e-9872-4030-8774-0d7c29bf0cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:15.979404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042293e-9872-4030-8774-0d7c29bf0cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:15.979868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042293e-9872-4030-8774-0d7c29bf0cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:15.980725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042293e-9872-4030-8774-0d7c29bf0cd5 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:15.982151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042293e-9872-4030-8774-0d7c29bf0cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:15.983609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042293e-9872-4030-8774-0d7c29bf0cd5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:15.985435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 61/127] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:15 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:15.991202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042293e-9872-4030-8774-0d7c29bf0cd5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:15.994092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac1b2c-b985-437e-aa2d-e21904da6bee None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:16.029777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aa274-9e92-4813-8c3c-ba1f295593b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:16.030721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aa274-9e92-4813-8c3c-ba1f295593b0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:16.031239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aa274-9e92-4813-8c3c-ba1f295593b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:16.031824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aa274-9e92-4813-8c3c-ba1f295593b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:16.032781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2aa274-9e92-4813-8c3c-ba1f295593b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:16.033280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2aa274-9e92-4813-8c3c-ba1f295593b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:16.035008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2aa274-9e92-4813-8c3c-ba1f295593b0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:16.050086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 67/128] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:16.051085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aa274-9e92-4813-8c3c-ba1f295593b0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:16.190274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042293e-9872-4030-8774-0d7c29bf0cd5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:16.213178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aa274-9e92-4813-8c3c-ba1f295593b0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:16.226344 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:16.254676 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa1fb9-7708-4cb8-b5ce-82d1c9222c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:16.255529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:16.256325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa1fb9-7708-4cb8-b5ce-82d1c9222c1d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:16.257039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa1fb9-7708-4cb8-b5ce-82d1c9222c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:16.257327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa1fb9-7708-4cb8-b5ce-82d1c9222c1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:16.270635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa1fb9-7708-4cb8-b5ce-82d1c9222c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:16.272458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583318a-b16e-4bb6-a009-5483055c0e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:16.273131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583318a-b16e-4bb6-a009-5483055c0e3c None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:16.273411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583318a-b16e-4bb6-a009-5483055c0e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:16.273686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583318a-b16e-4bb6-a009-5483055c0e3c None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/users/bb19e7f52e3f4b85bcc7952e23773462/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-43aa1fb9-7708-4cb8-b5ce-82d1c9222c1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:16.276437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:16.277315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:16.278130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 68/129] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:16.278787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa1fb9-7708-4cb8-b5ce-82d1c9222c1d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:16.280288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583318a-b16e-4bb6-a009-5483055c0e3c None admin] RBAC: Authorizing `identity:create_grant(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6, user_id=bb19e7f52e3f4b85bcc7952e23773462, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:16.281113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583318a-b16e-4bb6-a009-5483055c0e3c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:16.393000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 62/130] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:29:15 2022] PUT /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/users/bb19e7f52e3f4b85bcc7952e23773462/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:29:16.393649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583318a-b16e-4bb6-a009-5483055c0e3c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:16.404762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa1fb9-7708-4cb8-b5ce-82d1c9222c1d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:16.484832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df8123-c66e-4702-a42d-b9f57f767940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:16.485644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df8123-c66e-4702-a42d-b9f57f767940 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:16.485960 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df8123-c66e-4702-a42d-b9f57f767940 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:16.486248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df8123-c66e-4702-a42d-b9f57f767940 None admin] PATH_INFO: `/v3/users/c3e17b4e2c274478a5af0c474f6198f3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:16.489863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df8123-c66e-4702-a42d-b9f57f767940 None admin] RBAC: Authorizing `identity:get_user(user_id=c3e17b4e2c274478a5af0c474f6198f3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:16.491351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df8123-c66e-4702-a42d-b9f57f767940 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:16.495384 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 69/131] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:16 2022] GET /identity/v3/users/c3e17b4e2c274478a5af0c474f6198f3 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:16.501703 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df8123-c66e-4702-a42d-b9f57f767940 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:16.586534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:16.607589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583318a-b16e-4bb6-a009-5483055c0e3c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:16.688666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6c081-6a90-4aee-9eb4-715532c2c38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:16.689307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6c081-6a90-4aee-9eb4-715532c2c38a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:16.689592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6c081-6a90-4aee-9eb4-715532c2c38a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:16.689896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6c081-6a90-4aee-9eb4-715532c2c38a None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:16.690593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6c081-6a90-4aee-9eb4-715532c2c38a None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:16.690905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6c081-6a90-4aee-9eb4-715532c2c38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:16.692706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6c081-6a90-4aee-9eb4-715532c2c38a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:16.704181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 63/132] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:16.716830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6c081-6a90-4aee-9eb4-715532c2c38a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:16.741403 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df8123-c66e-4702-a42d-b9f57f767940 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:16.783489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eff3c6-188f-4754-b275-e3d13924064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:16.784252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eff3c6-188f-4754-b275-e3d13924064f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:16.784538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eff3c6-188f-4754-b275-e3d13924064f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:16.784818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eff3c6-188f-4754-b275-e3d13924064f None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:16.786026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eff3c6-188f-4754-b275-e3d13924064f None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:16.787675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eff3c6-188f-4754-b275-e3d13924064f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:16.789499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 70/133] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:16 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:16.790687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1fa69-3042-4166-9c41-3430f6f47bfa None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:16.790975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1fa69-3042-4166-9c41-3430f6f47bfa None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:16.791254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1fa69-3042-4166-9c41-3430f6f47bfa None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:16.792482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 71/134] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:16.801517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1fa69-3042-4166-9c41-3430f6f47bfa None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:16.813961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:16.912268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6c081-6a90-4aee-9eb4-715532c2c38a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:16.938352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1fa69-3042-4166-9c41-3430f6f47bfa None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:16.958221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:16.998938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ba4c9-1c2b-4cc4-ba55-38719b24f543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:16.998938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ba4c9-1c2b-4cc4-ba55-38719b24f543 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:16.998938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ba4c9-1c2b-4cc4-ba55-38719b24f543 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:17.000180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ba4c9-1c2b-4cc4-ba55-38719b24f543 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/users/c3e17b4e2c274478a5af0c474f6198f3/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:17.007168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85800b1-e53b-4846-9439-7529c7f978f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:17.008389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ba4c9-1c2b-4cc4-ba55-38719b24f543 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6, user_id=c3e17b4e2c274478a5af0c474f6198f3, role_id=ec3ad2e5f7314c3b8d35826cfd81b5ba)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:17.009426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85800b1-e53b-4846-9439-7529c7f978f9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:17.012646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85800b1-e53b-4846-9439-7529c7f978f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:17.013305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85800b1-e53b-4846-9439-7529c7f978f9 None admin] PATH_INFO: `/v3/users/c3e17b4e2c274478a5af0c474f6198f3` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:17.015134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85800b1-e53b-4846-9439-7529c7f978f9 None admin] RBAC: Authorizing `identity:get_user(user_id=c3e17b4e2c274478a5af0c474f6198f3)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:17.015519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ba4c9-1c2b-4cc4-ba55-38719b24f543 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:17.018412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85800b1-e53b-4846-9439-7529c7f978f9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:17.028765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 64/135] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:16 2022] GET /identity/v3/users/c3e17b4e2c274478a5af0c474f6198f3 => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:17.033899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af0d7e-58be-41a0-aacd-ef14f6e7013c None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:17.035206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af0d7e-58be-41a0-aacd-ef14f6e7013c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:17.035709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af0d7e-58be-41a0-aacd-ef14f6e7013c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:17.057820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-12af0d7e-58be-41a0-aacd-ef14f6e7013c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:17.059871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-12af0d7e-58be-41a0-aacd-ef14f6e7013c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:17.131155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 72/136] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:29:16 2022] PUT /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/users/c3e17b4e2c274478a5af0c474f6198f3/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:17.132831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ba4c9-1c2b-4cc4-ba55-38719b24f543 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:17.133886 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 65/137] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:17.288719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ba4c9-1c2b-4cc4-ba55-38719b24f543 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:17.332030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc5b3a-7747-49bb-bf28-085de6852f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:17.333039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc5b3a-7747-49bb-bf28-085de6852f5e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:17.333530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc5b3a-7747-49bb-bf28-085de6852f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:17.334054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc5b3a-7747-49bb-bf28-085de6852f5e None admin] PATH_INFO: `/v3/projects/58bf3c47d4214522bc44853d51d222db` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:17.335483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc5b3a-7747-49bb-bf28-085de6852f5e None admin] RBAC: Authorizing `identity:get_project(project_id=58bf3c47d4214522bc44853d51d222db)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:17.337260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc5b3a-7747-49bb-bf28-085de6852f5e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:17.339155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 73/138] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:17 2022] GET /identity/v3/projects/58bf3c47d4214522bc44853d51d222db => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:17.345511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af0d7e-58be-41a0-aacd-ef14f6e7013c None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:17.418851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:17.476141 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af0d7e-58be-41a0-aacd-ef14f6e7013c None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:17.512028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a51e5-6f68-421d-90d0-df81e75e81c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:17.512817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a51e5-6f68-421d-90d0-df81e75e81c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:17.513097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a51e5-6f68-421d-90d0-df81e75e81c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:17.513411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a51e5-6f68-421d-90d0-df81e75e81c5 None admin] PATH_INFO: `/v3/projects/58bf3c47d4214522bc44853d51d222db/users/c3e17b4e2c274478a5af0c474f6198f3/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:17.515386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a51e5-6f68-421d-90d0-df81e75e81c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=58bf3c47d4214522bc44853d51d222db, user_id=c3e17b4e2c274478a5af0c474f6198f3, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:17.516837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a51e5-6f68-421d-90d0-df81e75e81c5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:17.523802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc5b3a-7747-49bb-bf28-085de6852f5e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:17.564833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:17.599611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 66/139] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:29:17 2022] PUT /identity/v3/projects/58bf3c47d4214522bc44853d51d222db/users/c3e17b4e2c274478a5af0c474f6198f3/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:17.645914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc5b3a-7747-49bb-bf28-085de6852f5e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:17.692666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c55dd-97a1-44c8-8c48-5b43e18549db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:17.693328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c55dd-97a1-44c8-8c48-5b43e18549db None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:17.693618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c55dd-97a1-44c8-8c48-5b43e18549db None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:17.693926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c55dd-97a1-44c8-8c48-5b43e18549db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:17.695075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c55dd-97a1-44c8-8c48-5b43e18549db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:17.696557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c55dd-97a1-44c8-8c48-5b43e18549db None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:17.698603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 74/140] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:17.704177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a51e5-6f68-421d-90d0-df81e75e81c5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:17.869913 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a51e5-6f68-421d-90d0-df81e75e81c5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:17.884078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:17.910471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3dc35d-c3ae-4c45-af6a-fa54f951a29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:17.911151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dc35d-c3ae-4c45-af6a-fa54f951a29b None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:17.911438 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dc35d-c3ae-4c45-af6a-fa54f951a29b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:17.911711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dc35d-c3ae-4c45-af6a-fa54f951a29b None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:17.912376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3dc35d-c3ae-4c45-af6a-fa54f951a29b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:17.913872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3dc35d-c3ae-4c45-af6a-fa54f951a29b None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:18.044633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 67/141] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:29:17 2022] POST /identity/v3/users => generated 283 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:29:18.449405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298de84-90bd-4f3e-a59d-d4e13b977678 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:18.450022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298de84-90bd-4f3e-a59d-d4e13b977678 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:18.450318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298de84-90bd-4f3e-a59d-d4e13b977678 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:18.451707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 75/142] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:18.457458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9925c3-522f-4409-8259-66988ab2d266 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:18.457764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9925c3-522f-4409-8259-66988ab2d266 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:18.458049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9925c3-522f-4409-8259-66988ab2d266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:18.486108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-db9925c3-522f-4409-8259-66988ab2d266 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:18.487376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-db9925c3-522f-4409-8259-66988ab2d266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:18.578512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 68/143] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:18.642991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb3038-f6ad-4e32-8548-2e0d38b33c40 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:18.643311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb3038-f6ad-4e32-8548-2e0d38b33c40 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:18.643546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb3038-f6ad-4e32-8548-2e0d38b33c40 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:18.644926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 76/144] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:18.651120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb826600-35c0-4096-a6d4-5bdd4214067f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:18.651402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb826600-35c0-4096-a6d4-5bdd4214067f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:18.651695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb826600-35c0-4096-a6d4-5bdd4214067f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:18.677012 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-bb826600-35c0-4096-a6d4-5bdd4214067f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:18.678285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bb826600-35c0-4096-a6d4-5bdd4214067f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:18.764132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 69/145] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:18.904160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb3038-f6ad-4e32-8548-2e0d38b33c40 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:19.039127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb3038-f6ad-4e32-8548-2e0d38b33c40 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:19.041092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:19.082085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b382698-90ba-479a-97a8-dfe3bc8251cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:19.082782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b382698-90ba-479a-97a8-dfe3bc8251cb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:19.083062 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b382698-90ba-479a-97a8-dfe3bc8251cb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:19.083387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b382698-90ba-479a-97a8-dfe3bc8251cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:19.098645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b382698-90ba-479a-97a8-dfe3bc8251cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-5b382698-90ba-479a-97a8-dfe3bc8251cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:19.103608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:19.104442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:19.105812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 77/146] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:19.113002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb826600-35c0-4096-a6d4-5bdd4214067f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:19.148283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:19.179474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b382698-90ba-479a-97a8-dfe3bc8251cb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:19.239646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb826600-35c0-4096-a6d4-5bdd4214067f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:19.283312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9e206-b69b-48ea-b48a-c876810801b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:19.283986 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9e206-b69b-48ea-b48a-c876810801b2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:19.284269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9e206-b69b-48ea-b48a-c876810801b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:19.284541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9e206-b69b-48ea-b48a-c876810801b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:19.285213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9e206-b69b-48ea-b48a-c876810801b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:19.285514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9e206-b69b-48ea-b48a-c876810801b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:19.286935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9e206-b69b-48ea-b48a-c876810801b2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:19.295825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 70/147] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:19.301350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9e206-b69b-48ea-b48a-c876810801b2 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:19.395765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:19.407922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b382698-90ba-479a-97a8-dfe3bc8251cb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:19.408073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9e206-b69b-48ea-b48a-c876810801b2 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:19.442786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57353d-a15f-4c9d-9ee5-e1df5d4c563d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:19.443660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57353d-a15f-4c9d-9ee5-e1df5d4c563d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:19.444217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57353d-a15f-4c9d-9ee5-e1df5d4c563d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:19.444817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57353d-a15f-4c9d-9ee5-e1df5d4c563d None admin] PATH_INFO: `/v3/users/efb5876c14074177a961c3a7fd2032c5` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:19.446479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57353d-a15f-4c9d-9ee5-e1df5d4c563d None admin] RBAC: Authorizing `identity:get_user(user_id=efb5876c14074177a961c3a7fd2032c5)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:19.448069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57353d-a15f-4c9d-9ee5-e1df5d4c563d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:19.450223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 71/148] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:19 2022] GET /identity/v3/users/efb5876c14074177a961c3a7fd2032c5 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:19.455933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57353d-a15f-4c9d-9ee5-e1df5d4c563d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:19.478481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d60f8-87f1-4c4b-bb8d-c4c8569e43c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:19.479671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d60f8-87f1-4c4b-bb8d-c4c8569e43c5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:19.480147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d60f8-87f1-4c4b-bb8d-c4c8569e43c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:19.480560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d60f8-87f1-4c4b-bb8d-c4c8569e43c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:19.508679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d60f8-87f1-4c4b-bb8d-c4c8569e43c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a99d60f8-87f1-4c4b-bb8d-c4c8569e43c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:19.513620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:19.514616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:19.515389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:19.522903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 78/149] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:19.530487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d60f8-87f1-4c4b-bb8d-c4c8569e43c5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:19.577333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57353d-a15f-4c9d-9ee5-e1df5d4c563d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:19.599406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:19.619800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d1fbc-7cf9-4132-b476-0dc4232f56ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:19.620448 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d1fbc-7cf9-4132-b476-0dc4232f56ff None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:19.620963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d1fbc-7cf9-4132-b476-0dc4232f56ff None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:19.621314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d1fbc-7cf9-4132-b476-0dc4232f56ff None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:19.623382 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d1fbc-7cf9-4132-b476-0dc4232f56ff None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:19.624865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d1fbc-7cf9-4132-b476-0dc4232f56ff None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:19.629647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 72/150] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:19 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:19.636091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d1fbc-7cf9-4132-b476-0dc4232f56ff None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:19.682872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:19.754013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d60f8-87f1-4c4b-bb8d-c4c8569e43c5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:19.774080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d1fbc-7cf9-4132-b476-0dc4232f56ff None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:19.810102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3db43d-f760-4ce4-85c2-51079c3f2262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:19.810772 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3db43d-f760-4ce4-85c2-51079c3f2262 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:19.811054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3db43d-f760-4ce4-85c2-51079c3f2262 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:19.811396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3db43d-f760-4ce4-85c2-51079c3f2262 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/users/efb5876c14074177a961c3a7fd2032c5/roles/ed414442e2d24f2f99e36d88ade1835e` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:19.814039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3db43d-f760-4ce4-85c2-51079c3f2262 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6, user_id=efb5876c14074177a961c3a7fd2032c5, role_id=ed414442e2d24f2f99e36d88ade1835e)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:19.815662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3db43d-f760-4ce4-85c2-51079c3f2262 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:19.835107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c278897-573d-4984-86ed-b99d56cdc3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:19.835866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c278897-573d-4984-86ed-b99d56cdc3cc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:19.836253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c278897-573d-4984-86ed-b99d56cdc3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:19.836635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c278897-573d-4984-86ed-b99d56cdc3cc None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:19.837420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c278897-573d-4984-86ed-b99d56cdc3cc None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:19.837876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c278897-573d-4984-86ed-b99d56cdc3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:19.839379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c278897-573d-4984-86ed-b99d56cdc3cc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:19.852321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 79/151] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:19.856851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf0c21-27ac-49e7-97fc-b9d8e2c4ccdc None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:19.857276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf0c21-27ac-49e7-97fc-b9d8e2c4ccdc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:19.857654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf0c21-27ac-49e7-97fc-b9d8e2c4ccdc None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:19.859273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 80/152] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:29:19.865394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf0c21-27ac-49e7-97fc-b9d8e2c4ccdc None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:19.894437 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 73/153] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:29:19 2022] PUT /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/users/efb5876c14074177a961c3a7fd2032c5/roles/ed414442e2d24f2f99e36d88ade1835e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:29:19.896020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fe7b7-124c-4570-9888-14d81df4391b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:19.896446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fe7b7-124c-4570-9888-14d81df4391b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:19.896848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fe7b7-124c-4570-9888-14d81df4391b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:19.931013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-3d7fe7b7-124c-4570-9888-14d81df4391b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:19.932551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-3d7fe7b7-124c-4570-9888-14d81df4391b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:20.018100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:20.040847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 74/154] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:20.054590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf0c21-27ac-49e7-97fc-b9d8e2c4ccdc None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:20.115064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f312-4103-4f29-b789-18c6d22532a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:20.115650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f312-4103-4f29-b789-18c6d22532a8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:20.115935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f312-4103-4f29-b789-18c6d22532a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:20.116218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f312-4103-4f29-b789-18c6d22532a8 None admin] PATH_INFO: `/v3/users/c3e17b4e2c274478a5af0c474f6198f3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:20.118812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182f312-4103-4f29-b789-18c6d22532a8 None admin] RBAC: Authorizing `identity:get_user(user_id=c3e17b4e2c274478a5af0c474f6198f3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:20.120311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182f312-4103-4f29-b789-18c6d22532a8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:20.122644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 81/155] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:19 2022] GET /identity/v3/users/c3e17b4e2c274478a5af0c474f6198f3 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:20.128406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fe7b7-124c-4570-9888-14d81df4391b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:20.152460 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:20.276241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fe7b7-124c-4570-9888-14d81df4391b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:20.313598 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e213-10d8-4be4-bd3f-317b388294a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:20.314246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e213-10d8-4be4-bd3f-317b388294a2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:20.314516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e213-10d8-4be4-bd3f-317b388294a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:20.314791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e213-10d8-4be4-bd3f-317b388294a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:20.393257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832e213-10d8-4be4-bd3f-317b388294a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:20.394703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832e213-10d8-4be4-bd3f-317b388294a2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:20.411746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:20.466385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f312-4103-4f29-b789-18c6d22532a8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:20.504145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 75/156] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:20.510035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e213-10d8-4be4-bd3f-317b388294a2 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:20.615136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e213-10d8-4be4-bd3f-317b388294a2 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:20.630123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:20.670481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a848e-3804-49e9-bf66-59787011bdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:20.671135 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a848e-3804-49e9-bf66-59787011bdc2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:20.671410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a848e-3804-49e9-bf66-59787011bdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:20.671686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a848e-3804-49e9-bf66-59787011bdc2 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:20.672911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a848e-3804-49e9-bf66-59787011bdc2 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:20.674466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a848e-3804-49e9-bf66-59787011bdc2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:20.676183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 76/157] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:20 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:20.677313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d117a-a880-4036-acee-a9de0b642289 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:20.677592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d117a-a880-4036-acee-a9de0b642289 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:20.677889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d117a-a880-4036-acee-a9de0b642289 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:20.679206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 77/158] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:20.684795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1123f-2fe1-49f8-a314-a7e4016b6cdc None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:20.685073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1123f-2fe1-49f8-a314-a7e4016b6cdc None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:20.685344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1123f-2fe1-49f8-a314-a7e4016b6cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:20.715622 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-efe1123f-2fe1-49f8-a314-a7e4016b6cdc None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:20.717434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-efe1123f-2fe1-49f8-a314-a7e4016b6cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:20.743930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f312-4103-4f29-b789-18c6d22532a8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:20.788203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 78/159] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:20.789160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1123f-2fe1-49f8-a314-a7e4016b6cdc None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:20.812593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b9432-ae63-468d-8b64-7b36eed9ff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:20.813238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b9432-ae63-468d-8b64-7b36eed9ff2b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:20.813522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b9432-ae63-468d-8b64-7b36eed9ff2b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:20.813910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b9432-ae63-468d-8b64-7b36eed9ff2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:20.834836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b9432-ae63-468d-8b64-7b36eed9ff2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c49b9432-ae63-468d-8b64-7b36eed9ff2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:20.844466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:20.845401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:20.846305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 82/160] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:29:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:20.854645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b9432-ae63-468d-8b64-7b36eed9ff2b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:20.875893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:20.893125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1123f-2fe1-49f8-a314-a7e4016b6cdc None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:20.910093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:20.929451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252fd86-5dc6-472c-a668-21df91d98337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:20.930152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252fd86-5dc6-472c-a668-21df91d98337 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:20.930596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252fd86-5dc6-472c-a668-21df91d98337 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:20.930985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252fd86-5dc6-472c-a668-21df91d98337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:20.931858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252fd86-5dc6-472c-a668-21df91d98337 None admin] RBAC: Adding query filter params (role.id=d801ebea910445a7a6100b35e07630dd, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6, user.id=c3e17b4e2c274478a5af0c474f6198f3) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:20.932262 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252fd86-5dc6-472c-a668-21df91d98337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:20.937765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252fd86-5dc6-472c-a668-21df91d98337 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:20.940770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 79/161] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:20 2022] GET /identity/v3/role_assignments?user.id=c3e17b4e2c274478a5af0c474f6198f3&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=d801ebea910445a7a6100b35e07630dd => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:21.000772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b9432-ae63-468d-8b64-7b36eed9ff2b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:21.046392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14186a-eccf-42f0-852e-efffe9e37168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:21.047269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14186a-eccf-42f0-852e-efffe9e37168 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:21.047697 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14186a-eccf-42f0-852e-efffe9e37168 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:21.048141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14186a-eccf-42f0-852e-efffe9e37168 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:21.049024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14186a-eccf-42f0-852e-efffe9e37168 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:21.049475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14186a-eccf-42f0-852e-efffe9e37168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:21.051017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14186a-eccf-42f0-852e-efffe9e37168 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:21.060690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 83/162] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:21.066712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252fd86-5dc6-472c-a668-21df91d98337 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:21.258657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252fd86-5dc6-472c-a668-21df91d98337 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:21.303745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2058d4d-cd4b-4d4b-91e5-48e78f22f475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:21.304379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2058d4d-cd4b-4d4b-91e5-48e78f22f475 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:21.304653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2058d4d-cd4b-4d4b-91e5-48e78f22f475 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:21.304921 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2058d4d-cd4b-4d4b-91e5-48e78f22f475 None admin] PATH_INFO: `/v3/users/bb19e7f52e3f4b85bcc7952e23773462` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:21.310516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2058d4d-cd4b-4d4b-91e5-48e78f22f475 None admin] RBAC: Authorizing `identity:get_user(user_id=bb19e7f52e3f4b85bcc7952e23773462)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:21.311990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2058d4d-cd4b-4d4b-91e5-48e78f22f475 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:21.314044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 80/163] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:21 2022] GET /identity/v3/users/bb19e7f52e3f4b85bcc7952e23773462 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:21.319652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14186a-eccf-42f0-852e-efffe9e37168 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:21.375790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2058d4d-cd4b-4d4b-91e5-48e78f22f475 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:21.424962 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:21.451900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14186a-eccf-42f0-852e-efffe9e37168 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:21.465291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:21.496451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba83794-3b76-42a0-8b60-c280af2d67a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:21.498630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba83794-3b76-42a0-8b60-c280af2d67a3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:21.499832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba83794-3b76-42a0-8b60-c280af2d67a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:21.500941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba83794-3b76-42a0-8b60-c280af2d67a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:21.593751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba83794-3b76-42a0-8b60-c280af2d67a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:21.595589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba83794-3b76-42a0-8b60-c280af2d67a3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:21.613759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2058d4d-cd4b-4d4b-91e5-48e78f22f475 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:21.692762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc3d77-18c6-4dba-a80e-aebaffa4a451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:21.693407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc3d77-18c6-4dba-a80e-aebaffa4a451 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:21.693683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc3d77-18c6-4dba-a80e-aebaffa4a451 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:21.693979 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc3d77-18c6-4dba-a80e-aebaffa4a451 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:21.705687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc3d77-18c6-4dba-a80e-aebaffa4a451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:21.711210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-efcc3d77-18c6-4dba-a80e-aebaffa4a451 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:21.720800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:21.723553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:21.724544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 81/164] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:21.728105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc3d77-18c6-4dba-a80e-aebaffa4a451 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:21.758192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 84/165] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:21.783209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba83794-3b76-42a0-8b60-c280af2d67a3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:21.927938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc3d77-18c6-4dba-a80e-aebaffa4a451 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:21.940917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:22.006423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba83794-3b76-42a0-8b60-c280af2d67a3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:22.033902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40043-ecb6-4d4b-ae58-5b1583f6dfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:22.034516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40043-ecb6-4d4b-ae58-5b1583f6dfdd None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:22.034795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40043-ecb6-4d4b-ae58-5b1583f6dfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:22.035060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40043-ecb6-4d4b-ae58-5b1583f6dfdd None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:22.035735 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40043-ecb6-4d4b-ae58-5b1583f6dfdd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:22.036034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40043-ecb6-4d4b-ae58-5b1583f6dfdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:22.042379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40043-ecb6-4d4b-ae58-5b1583f6dfdd None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:22.061162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 82/166] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:22.062428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913cee6-bb96-4f37-b633-6c2df09ad9dd None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:22.062704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913cee6-bb96-4f37-b633-6c2df09ad9dd None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:22.062977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913cee6-bb96-4f37-b633-6c2df09ad9dd None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:22.067241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 83/167] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:22.070004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913cee6-bb96-4f37-b633-6c2df09ad9dd None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:22.085904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcef66-4120-40bd-b2b1-08c9609f8fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:22.086558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcef66-4120-40bd-b2b1-08c9609f8fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:22.086890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcef66-4120-40bd-b2b1-08c9609f8fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:22.087171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcef66-4120-40bd-b2b1-08c9609f8fd3 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:22.088364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcef66-4120-40bd-b2b1-08c9609f8fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:22.089946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcef66-4120-40bd-b2b1-08c9609f8fd3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:22.091666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 85/168] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:21 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:22.099006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e484584-6043-4450-bc00-01b030b1199b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:22.099594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e484584-6043-4450-bc00-01b030b1199b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:22.099902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e484584-6043-4450-bc00-01b030b1199b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:22.129306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1e484584-6043-4450-bc00-01b030b1199b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:22.130882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1e484584-6043-4450-bc00-01b030b1199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:22.196044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:22.252725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 86/169] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:22.253561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e484584-6043-4450-bc00-01b030b1199b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:22.384712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913cee6-bb96-4f37-b633-6c2df09ad9dd None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:22.415473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e484584-6043-4450-bc00-01b030b1199b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:22.456784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51ebff-0f42-4897-a507-ee7f33cf8451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:22.457323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51ebff-0f42-4897-a507-ee7f33cf8451 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:22.457601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51ebff-0f42-4897-a507-ee7f33cf8451 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:22.457902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51ebff-0f42-4897-a507-ee7f33cf8451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:22.458649 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51ebff-0f42-4897-a507-ee7f33cf8451 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6, user.id=bb19e7f52e3f4b85bcc7952e23773462) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:22.458985 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51ebff-0f42-4897-a507-ee7f33cf8451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:22.461126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51ebff-0f42-4897-a507-ee7f33cf8451 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:22.469599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e301-b71c-47d3-a007-dd5d9cd782b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:22.470316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e301-b71c-47d3-a007-dd5d9cd782b2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:22.470603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e301-b71c-47d3-a007-dd5d9cd782b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:22.470878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e301-b71c-47d3-a007-dd5d9cd782b2 None admin] PATH_INFO: `/v3/users/c3e17b4e2c274478a5af0c474f6198f3` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:22.472279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9e301-b71c-47d3-a007-dd5d9cd782b2 None admin] RBAC: Authorizing `identity:get_user(user_id=c3e17b4e2c274478a5af0c474f6198f3)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:22.474942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9e301-b71c-47d3-a007-dd5d9cd782b2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:22.476551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 84/170] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:22 2022] GET /identity/v3/users/c3e17b4e2c274478a5af0c474f6198f3 => generated 275 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:22.477486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 87/171] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:22 2022] GET /identity/v3/role_assignments?user.id=bb19e7f52e3f4b85bcc7952e23773462&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:22.490749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e301-b71c-47d3-a007-dd5d9cd782b2 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:22.534140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:22.624252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e301-b71c-47d3-a007-dd5d9cd782b2 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:22.664611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201e626-7086-4e19-afda-1c2dac2df747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:22.665226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201e626-7086-4e19-afda-1c2dac2df747 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:22.665504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201e626-7086-4e19-afda-1c2dac2df747 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:22.665802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201e626-7086-4e19-afda-1c2dac2df747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:22.671473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51ebff-0f42-4897-a507-ee7f33cf8451 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:22.732028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:22.753172 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201e626-7086-4e19-afda-1c2dac2df747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:22.754820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201e626-7086-4e19-afda-1c2dac2df747 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:22.791766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:22.875958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51ebff-0f42-4897-a507-ee7f33cf8451 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:22.914082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 85/172] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:22.920354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201e626-7086-4e19-afda-1c2dac2df747 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:22.962623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0229a8a-1207-4162-a553-1264efc88f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:22.963280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0229a8a-1207-4162-a553-1264efc88f02 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:22.963745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0229a8a-1207-4162-a553-1264efc88f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:22.964175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0229a8a-1207-4162-a553-1264efc88f02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:22.998945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0229a8a-1207-4162-a553-1264efc88f02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a0229a8a-1207-4162-a553-1264efc88f02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:23.004249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:23.005427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:23.006510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:23.008530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 88/173] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:23.015450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0229a8a-1207-4162-a553-1264efc88f02 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:23.156824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201e626-7086-4e19-afda-1c2dac2df747 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:23.179895 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:23.187946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0229a8a-1207-4162-a553-1264efc88f02 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:23.223659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec53ddf-6904-4350-a55e-2a139698b9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:23.224293 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec53ddf-6904-4350-a55e-2a139698b9af None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:23.224565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec53ddf-6904-4350-a55e-2a139698b9af None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:23.224831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec53ddf-6904-4350-a55e-2a139698b9af None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:23.227094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec53ddf-6904-4350-a55e-2a139698b9af None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:23.228539 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec53ddf-6904-4350-a55e-2a139698b9af None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:23.236774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 86/174] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:22 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:23.238134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b8a29-5bac-4fbf-8f62-e71555212567 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:23.238414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b8a29-5bac-4fbf-8f62-e71555212567 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:23.238682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b8a29-5bac-4fbf-8f62-e71555212567 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:23.242326 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 87/175] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:23.254351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b8a29-5bac-4fbf-8f62-e71555212567 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:23.257604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:23.260085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbfc8f-2b15-429a-b89e-7b085fa82565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:23.260085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbfc8f-2b15-429a-b89e-7b085fa82565 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:23.260085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbfc8f-2b15-429a-b89e-7b085fa82565 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:23.260085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbfc8f-2b15-429a-b89e-7b085fa82565 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:23.260795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbfc8f-2b15-429a-b89e-7b085fa82565 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:23.261104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbfc8f-2b15-429a-b89e-7b085fa82565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:23.267087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbfc8f-2b15-429a-b89e-7b085fa82565 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:23.284313 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 89/176] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:23.285583 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bb733-79f8-4687-b276-95fb82bdf517 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:23.285891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bb733-79f8-4687-b276-95fb82bdf517 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:23.286166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bb733-79f8-4687-b276-95fb82bdf517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:23.319565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-428bb733-79f8-4687-b276-95fb82bdf517 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:23.321544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-428bb733-79f8-4687-b276-95fb82bdf517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:23.417015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b8a29-5bac-4fbf-8f62-e71555212567 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:23.467097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8e95d-9ee7-4462-9b39-cad34c4c153d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:23.468067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8e95d-9ee7-4462-9b39-cad34c4c153d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:23.468544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8e95d-9ee7-4462-9b39-cad34c4c153d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:23.469077 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8e95d-9ee7-4462-9b39-cad34c4c153d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:23.470158 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8e95d-9ee7-4462-9b39-cad34c4c153d None admin] RBAC: Adding query filter params (role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6, user.id=c3e17b4e2c274478a5af0c474f6198f3) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:23.470686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8e95d-9ee7-4462-9b39-cad34c4c153d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:23.472465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8e95d-9ee7-4462-9b39-cad34c4c153d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:23.479615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 88/177] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:23 2022] GET /identity/v3/role_assignments?user.id=c3e17b4e2c274478a5af0c474f6198f3&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:23.485752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8e95d-9ee7-4462-9b39-cad34c4c153d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:23.523937 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 90/178] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:23.591434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:23.643459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8e95d-9ee7-4462-9b39-cad34c4c153d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:23.692701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500e139-442c-41ef-a258-d3a0bb91d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:23.693340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500e139-442c-41ef-a258-d3a0bb91d6df None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:23.693614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500e139-442c-41ef-a258-d3a0bb91d6df None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:23.693923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500e139-442c-41ef-a258-d3a0bb91d6df None admin] PATH_INFO: `/v3/users/c3e17b4e2c274478a5af0c474f6198f3` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:23.695055 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500e139-442c-41ef-a258-d3a0bb91d6df None admin] RBAC: Authorizing `identity:get_user(user_id=c3e17b4e2c274478a5af0c474f6198f3)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:23.696494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500e139-442c-41ef-a258-d3a0bb91d6df None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:23.701836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 89/179] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:23 2022] GET /identity/v3/users/c3e17b4e2c274478a5af0c474f6198f3 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:23.706896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bb733-79f8-4687-b276-95fb82bdf517 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:23.843582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:23.855852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bb733-79f8-4687-b276-95fb82bdf517 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:23.893400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d011eb7-3718-424e-a88b-adb93ed59e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:23.894076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d011eb7-3718-424e-a88b-adb93ed59e78 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:23.894322 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d011eb7-3718-424e-a88b-adb93ed59e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:23.894583 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d011eb7-3718-424e-a88b-adb93ed59e78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:23.912567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1164eab-81bc-4201-b847-15bdb3482290 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:23.912891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1164eab-81bc-4201-b847-15bdb3482290 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:23.913111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1164eab-81bc-4201-b847-15bdb3482290 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:23.915724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 90/180] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:23.920625 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17acdf-d014-4832-98b8-214fbc4f359f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:23.920905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17acdf-d014-4832-98b8-214fbc4f359f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:23.921177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17acdf-d014-4832-98b8-214fbc4f359f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:23.961809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4c17acdf-d014-4832-98b8-214fbc4f359f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:23.963000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4c17acdf-d014-4832-98b8-214fbc4f359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:23.972249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d011eb7-3718-424e-a88b-adb93ed59e78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:23.973753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d011eb7-3718-424e-a88b-adb93ed59e78 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:24.053485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 91/181] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:24.055133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77ff3d-0187-495c-b760-019310b8293d None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.055511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77ff3d-0187-495c-b760-019310b8293d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.055858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77ff3d-0187-495c-b760-019310b8293d None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.057132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 92/182] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:24.061792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77ff3d-0187-495c-b760-019310b8293d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:24.078397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:24.098815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 91/183] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:24.100291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da9ece-1729-47cb-81ac-41066701f306 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.100588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da9ece-1729-47cb-81ac-41066701f306 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.100867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da9ece-1729-47cb-81ac-41066701f306 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.102335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 92/184] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:24.103376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e9c5d-f4ea-48fe-8626-52ead580d450 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.103839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e9c5d-f4ea-48fe-8626-52ead580d450 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.104131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e9c5d-f4ea-48fe-8626-52ead580d450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.125248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b87e9c5d-f4ea-48fe-8626-52ead580d450 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:24.126455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b87e9c5d-f4ea-48fe-8626-52ead580d450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:24.134977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:24.176946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77ff3d-0187-495c-b760-019310b8293d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:24.201761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 93/185] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:24.201761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15342111-f7a8-4c4e-8f31-49ca3985fa1f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.201761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15342111-f7a8-4c4e-8f31-49ca3985fa1f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.201761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15342111-f7a8-4c4e-8f31-49ca3985fa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.212129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca92b0-4cc2-49dd-a375-db703493f8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:24.212755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca92b0-4cc2-49dd-a375-db703493f8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.213035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca92b0-4cc2-49dd-a375-db703493f8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.213302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca92b0-4cc2-49dd-a375-db703493f8f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.223957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-15342111-f7a8-4c4e-8f31-49ca3985fa1f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:24.224132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca92b0-4cc2-49dd-a375-db703493f8f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:24.225131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-15342111-f7a8-4c4e-8f31-49ca3985fa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-5aca92b0-4cc2-49dd-a375-db703493f8f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:24.229013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:24.230474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:24.231289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 93/186] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:29:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:24.231948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca92b0-4cc2-49dd-a375-db703493f8f9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:24.328571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 94/187] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:24.329251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15342111-f7a8-4c4e-8f31-49ca3985fa1f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:24.344360 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca92b0-4cc2-49dd-a375-db703493f8f9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:24.370457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:24.383074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34214ce-6580-4325-93c5-7d140003fe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:24.383708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34214ce-6580-4325-93c5-7d140003fe3c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.384117 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34214ce-6580-4325-93c5-7d140003fe3c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.384401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34214ce-6580-4325-93c5-7d140003fe3c None admin] PATH_INFO: `/v3/projects/58bf3c47d4214522bc44853d51d222db` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.385562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34214ce-6580-4325-93c5-7d140003fe3c None admin] RBAC: Authorizing `identity:get_project(project_id=58bf3c47d4214522bc44853d51d222db)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:24.387977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34214ce-6580-4325-93c5-7d140003fe3c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:24.397154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 94/188] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:24 2022] GET /identity/v3/projects/58bf3c47d4214522bc44853d51d222db => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:24.397154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34214ce-6580-4325-93c5-7d140003fe3c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:24.397461 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:24.460858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15342111-f7a8-4c4e-8f31-49ca3985fa1f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:24.502065 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34214ce-6580-4325-93c5-7d140003fe3c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:24.507714 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e64070-c57c-4fcb-bfb4-b8f590f9c094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:24.508642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e64070-c57c-4fcb-bfb4-b8f590f9c094 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.509618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e64070-c57c-4fcb-bfb4-b8f590f9c094 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.510076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e64070-c57c-4fcb-bfb4-b8f590f9c094 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.510879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e64070-c57c-4fcb-bfb4-b8f590f9c094 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:24.512365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e64070-c57c-4fcb-bfb4-b8f590f9c094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:24.514607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e64070-c57c-4fcb-bfb4-b8f590f9c094 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:24.530273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 95/189] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:24.530273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e64070-c57c-4fcb-bfb4-b8f590f9c094 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:24.541893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf932b-d9f7-4b11-a05c-4eb80b74c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:24.543150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf932b-d9f7-4b11-a05c-4eb80b74c53f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.543435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf932b-d9f7-4b11-a05c-4eb80b74c53f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.543806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf932b-d9f7-4b11-a05c-4eb80b74c53f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.544829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf932b-d9f7-4b11-a05c-4eb80b74c53f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:24.546463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf932b-d9f7-4b11-a05c-4eb80b74c53f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:24.548052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 95/190] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:24.548678 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf932b-d9f7-4b11-a05c-4eb80b74c53f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:24.614520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:24.632873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e64070-c57c-4fcb-bfb4-b8f590f9c094 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:24.643680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:24.655603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf932b-d9f7-4b11-a05c-4eb80b74c53f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:24.666864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd2c7b-da64-4365-8585-6b64eb014292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:24.668080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2c7b-da64-4365-8585-6b64eb014292 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.668424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2c7b-da64-4365-8585-6b64eb014292 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.668693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2c7b-da64-4365-8585-6b64eb014292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.669438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd2c7b-da64-4365-8585-6b64eb014292 None admin] RBAC: Adding query filter params (role.id=d801ebea910445a7a6100b35e07630dd, scope.project.id=58bf3c47d4214522bc44853d51d222db, user.id=c3e17b4e2c274478a5af0c474f6198f3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:24.669763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd2c7b-da64-4365-8585-6b64eb014292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:24.671251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd2c7b-da64-4365-8585-6b64eb014292 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:24.677407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 96/191] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:24 2022] GET /identity/v3/role_assignments?user.id=c3e17b4e2c274478a5af0c474f6198f3&scope.project.id=58bf3c47d4214522bc44853d51d222db&role.id=d801ebea910445a7a6100b35e07630dd => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:24.678204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd2c7b-da64-4365-8585-6b64eb014292 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:24.698938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5928f8-9974-4504-9bdc-9ee811e2847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:24.699582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5928f8-9974-4504-9bdc-9ee811e2847c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.699858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5928f8-9974-4504-9bdc-9ee811e2847c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.700126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5928f8-9974-4504-9bdc-9ee811e2847c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.713794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5928f8-9974-4504-9bdc-9ee811e2847c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-cd5928f8-9974-4504-9bdc-9ee811e2847c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:24.720311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:24.721302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:24.722191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 96/192] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:24.722883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5928f8-9974-4504-9bdc-9ee811e2847c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:24.817446 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd2c7b-da64-4365-8585-6b64eb014292 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:24.844558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5928f8-9974-4504-9bdc-9ee811e2847c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:24.857176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5c8ed-65a0-4059-8a5d-5fe0c42e6e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:24.858142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5c8ed-65a0-4059-8a5d-5fe0c42e6e28 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.859108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5c8ed-65a0-4059-8a5d-5fe0c42e6e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.859108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5c8ed-65a0-4059-8a5d-5fe0c42e6e28 None admin] PATH_INFO: `/v3/users/660d48ed2ad942859b0cf43abaa925aa` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.867980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:24.876299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5c8ed-65a0-4059-8a5d-5fe0c42e6e28 None admin] RBAC: Authorizing `identity:get_user(user_id=660d48ed2ad942859b0cf43abaa925aa)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:24.882174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5c8ed-65a0-4059-8a5d-5fe0c42e6e28 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:24.882174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 97/193] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:24 2022] GET /identity/v3/users/660d48ed2ad942859b0cf43abaa925aa => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:24.882174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5c8ed-65a0-4059-8a5d-5fe0c42e6e28 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:24.896129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57efa1e-7d3c-452f-8646-e417faa453e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:24.896746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57efa1e-7d3c-452f-8646-e417faa453e9 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:24.897026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57efa1e-7d3c-452f-8646-e417faa453e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:24.897301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57efa1e-7d3c-452f-8646-e417faa453e9 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:24.898005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57efa1e-7d3c-452f-8646-e417faa453e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:24.899522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57efa1e-7d3c-452f-8646-e417faa453e9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:24.920135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:24.987226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5c8ed-65a0-4059-8a5d-5fe0c42e6e28 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:25.007503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 97/194] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:29:24 2022] POST /identity/v3/users => generated 297 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:29:25.008401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57efa1e-7d3c-452f-8646-e417faa453e9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:25.021764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800cd41-aebf-4c7a-8fb3-ae0620541ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:25.022417 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800cd41-aebf-4c7a-8fb3-ae0620541ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:25.022705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800cd41-aebf-4c7a-8fb3-ae0620541ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:25.022987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800cd41-aebf-4c7a-8fb3-ae0620541ee4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:25.040671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800cd41-aebf-4c7a-8fb3-ae0620541ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-6800cd41-aebf-4c7a-8fb3-ae0620541ee4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:25.046540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:25.048689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 98/195] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:25.049645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800cd41-aebf-4c7a-8fb3-ae0620541ee4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:25.159688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57efa1e-7d3c-452f-8646-e417faa453e9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:25.171843 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:25.181519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800cd41-aebf-4c7a-8fb3-ae0620541ee4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:25.199954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983b3e2-ffb2-42fe-b7d2-ef9735519a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:25.200589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983b3e2-ffb2-42fe-b7d2-ef9735519a8a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:25.200863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983b3e2-ffb2-42fe-b7d2-ef9735519a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:25.201130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983b3e2-ffb2-42fe-b7d2-ef9735519a8a None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:25.201827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983b3e2-ffb2-42fe-b7d2-ef9735519a8a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:25.202130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983b3e2-ffb2-42fe-b7d2-ef9735519a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:25.203497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983b3e2-ffb2-42fe-b7d2-ef9735519a8a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:25.212220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 98/196] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:25.213029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983b3e2-ffb2-42fe-b7d2-ef9735519a8a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:25.228919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0570c-d468-49c5-8da1-b9f33755e0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:25.229542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0570c-d468-49c5-8da1-b9f33755e0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:25.229883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0570c-d468-49c5-8da1-b9f33755e0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:25.230289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0570c-d468-49c5-8da1-b9f33755e0f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:25.256513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:25.320441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0570c-d468-49c5-8da1-b9f33755e0f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:25.322802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0570c-d468-49c5-8da1-b9f33755e0f3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:25.340818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983b3e2-ffb2-42fe-b7d2-ef9735519a8a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:25.385685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e9298-60d1-495b-bcbc-3881339d422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:25.388636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e9298-60d1-495b-bcbc-3881339d422f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:25.388636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e9298-60d1-495b-bcbc-3881339d422f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:25.388636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e9298-60d1-495b-bcbc-3881339d422f None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:25.388636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e9298-60d1-495b-bcbc-3881339d422f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:25.388946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e9298-60d1-495b-bcbc-3881339d422f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:25.390144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e9298-60d1-495b-bcbc-3881339d422f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:25.399831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 99/197] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:25.400675 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e9298-60d1-495b-bcbc-3881339d422f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:25.446127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:25.454485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 99/198] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:25.455327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0570c-d468-49c5-8da1-b9f33755e0f3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:25.508279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:25.511289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e9298-60d1-495b-bcbc-3881339d422f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:25.547699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91e5fc-f0ee-4a61-8601-c67c2b87c25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:25.548350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91e5fc-f0ee-4a61-8601-c67c2b87c25a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:25.548631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91e5fc-f0ee-4a61-8601-c67c2b87c25a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:25.548909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91e5fc-f0ee-4a61-8601-c67c2b87c25a None admin] PATH_INFO: `/v3/users/efb5876c14074177a961c3a7fd2032c5` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:25.550104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91e5fc-f0ee-4a61-8601-c67c2b87c25a None admin] RBAC: Authorizing `identity:get_user(user_id=efb5876c14074177a961c3a7fd2032c5)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:25.551565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91e5fc-f0ee-4a61-8601-c67c2b87c25a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:25.553173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 100/199] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:25 2022] GET /identity/v3/users/efb5876c14074177a961c3a7fd2032c5 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:25.554064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91e5fc-f0ee-4a61-8601-c67c2b87c25a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:25.566685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0570c-d468-49c5-8da1-b9f33755e0f3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:25.603959 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85acb2-4f42-4664-969d-b404feb7a536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:25.604642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85acb2-4f42-4664-969d-b404feb7a536 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:25.605500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85acb2-4f42-4664-969d-b404feb7a536 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:25.605810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85acb2-4f42-4664-969d-b404feb7a536 None admin] PATH_INFO: `/v3/users/660d48ed2ad942859b0cf43abaa925aa` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:25.607325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85acb2-4f42-4664-969d-b404feb7a536 None admin] RBAC: Authorizing `identity:get_user(user_id=660d48ed2ad942859b0cf43abaa925aa)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:25.608887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85acb2-4f42-4664-969d-b404feb7a536 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:25.610946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 100/200] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:25 2022] GET /identity/v3/users/660d48ed2ad942859b0cf43abaa925aa => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:25.611796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85acb2-4f42-4664-969d-b404feb7a536 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:25.660646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91e5fc-f0ee-4a61-8601-c67c2b87c25a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:25.690152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:25.697230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9760cf-3be9-4f0b-8de7-c823fd7fabb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:25.697832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9760cf-3be9-4f0b-8de7-c823fd7fabb5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:25.698535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9760cf-3be9-4f0b-8de7-c823fd7fabb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:25.698847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9760cf-3be9-4f0b-8de7-c823fd7fabb5 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:25.700174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9760cf-3be9-4f0b-8de7-c823fd7fabb5 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:25.701645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9760cf-3be9-4f0b-8de7-c823fd7fabb5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:25.703681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 101/201] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:25 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:25.704520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9760cf-3be9-4f0b-8de7-c823fd7fabb5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:25.721509 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85acb2-4f42-4664-969d-b404feb7a536 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:25.746444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:25.757368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42557d9d-a117-4dad-bbdb-8ff454c0ffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:25.758029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42557d9d-a117-4dad-bbdb-8ff454c0ffd7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:25.758306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42557d9d-a117-4dad-bbdb-8ff454c0ffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:25.758587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42557d9d-a117-4dad-bbdb-8ff454c0ffd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:25.814998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9760cf-3be9-4f0b-8de7-c823fd7fabb5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:25.836202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42557d9d-a117-4dad-bbdb-8ff454c0ffd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:25.837752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42557d9d-a117-4dad-bbdb-8ff454c0ffd7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:25.860052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f5b83-25d0-4701-a4e4-909f1118ecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:25.860771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f5b83-25d0-4701-a4e4-909f1118ecfe None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:25.861114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f5b83-25d0-4701-a4e4-909f1118ecfe None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:25.861451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f5b83-25d0-4701-a4e4-909f1118ecfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:25.908331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:25.931044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f5b83-25d0-4701-a4e4-909f1118ecfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:25.932452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f5b83-25d0-4701-a4e4-909f1118ecfe None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:25.955208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 101/202] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:25.956814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42557d9d-a117-4dad-bbdb-8ff454c0ffd7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:25.978132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:26.039982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 102/203] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:26.040909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f5b83-25d0-4701-a4e4-909f1118ecfe None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:26.065510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42557d9d-a117-4dad-bbdb-8ff454c0ffd7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:26.106187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f38b8-ccb8-470d-898b-3b7f482d9c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:26.107132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f38b8-ccb8-470d-898b-3b7f482d9c99 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:26.107591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f38b8-ccb8-470d-898b-3b7f482d9c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:26.108054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f38b8-ccb8-470d-898b-3b7f482d9c99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:26.108964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f38b8-ccb8-470d-898b-3b7f482d9c99 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=e02b0fbedb42442bab7a31b8db209acb, user.id=660d48ed2ad942859b0cf43abaa925aa) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:26.109470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f38b8-ccb8-470d-898b-3b7f482d9c99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:26.111150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f38b8-ccb8-470d-898b-3b7f482d9c99 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:26.117708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 102/204] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:25 2022] GET /identity/v3/role_assignments?user.id=660d48ed2ad942859b0cf43abaa925aa&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:26.119023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f38b8-ccb8-470d-898b-3b7f482d9c99 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:26.148687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:26.155832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f5b83-25d0-4701-a4e4-909f1118ecfe None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:26.221602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb092e-dc09-441b-a806-82cbccf23949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:26.222232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb092e-dc09-441b-a806-82cbccf23949 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:26.222509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb092e-dc09-441b-a806-82cbccf23949 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:26.222778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb092e-dc09-441b-a806-82cbccf23949 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:26.226519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb092e-dc09-441b-a806-82cbccf23949 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:26.228126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb092e-dc09-441b-a806-82cbccf23949 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:26.230178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 103/205] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:26 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:26.236019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb092e-dc09-441b-a806-82cbccf23949 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:26.237370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f38b8-ccb8-470d-898b-3b7f482d9c99 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:26.250151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:26.275728 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f1131-559e-4e6d-b2a5-891af0b257f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:26.284264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f1131-559e-4e6d-b2a5-891af0b257f7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:26.284264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f1131-559e-4e6d-b2a5-891af0b257f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:26.284264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f1131-559e-4e6d-b2a5-891af0b257f7 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:26.284264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f1131-559e-4e6d-b2a5-891af0b257f7 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:26.284264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f1131-559e-4e6d-b2a5-891af0b257f7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:26.284264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 103/206] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:26 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:26.295199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f1131-559e-4e6d-b2a5-891af0b257f7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:26.361509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb092e-dc09-441b-a806-82cbccf23949 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:26.403324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865959d-0e7b-4dce-b1c0-22bf00329ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:26.403948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865959d-0e7b-4dce-b1c0-22bf00329ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:26.404219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865959d-0e7b-4dce-b1c0-22bf00329ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:26.404489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865959d-0e7b-4dce-b1c0-22bf00329ef6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:26.406073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865959d-0e7b-4dce-b1c0-22bf00329ef6 None admin] RBAC: Adding query filter params (role.id=ed414442e2d24f2f99e36d88ade1835e, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6, user.id=efb5876c14074177a961c3a7fd2032c5) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:26.407929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865959d-0e7b-4dce-b1c0-22bf00329ef6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:26.409480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865959d-0e7b-4dce-b1c0-22bf00329ef6 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:26.418658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 104/207] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:26 2022] GET /identity/v3/role_assignments?user.id=efb5876c14074177a961c3a7fd2032c5&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=ed414442e2d24f2f99e36d88ade1835e => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:26.470021 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:26.481260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f1131-559e-4e6d-b2a5-891af0b257f7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:26.554548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b9ed7-5c2d-4a84-aa12-7c2d4ae50308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:26.555363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b9ed7-5c2d-4a84-aa12-7c2d4ae50308 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:26.555731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b9ed7-5c2d-4a84-aa12-7c2d4ae50308 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:26.556100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b9ed7-5c2d-4a84-aa12-7c2d4ae50308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:26.556937 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b9ed7-5c2d-4a84-aa12-7c2d4ae50308 None admin] RBAC: Adding query filter params (role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba, scope.project.id=e02b0fbedb42442bab7a31b8db209acb, user.id=660d48ed2ad942859b0cf43abaa925aa) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:26.557341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b9ed7-5c2d-4a84-aa12-7c2d4ae50308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:26.559109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b9ed7-5c2d-4a84-aa12-7c2d4ae50308 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:26.570480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 104/208] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:26 2022] GET /identity/v3/role_assignments?user.id=660d48ed2ad942859b0cf43abaa925aa&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:27.427437 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f8c17-bd05-4ace-af4a-2b13453d4f8f None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:27.427996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f8c17-bd05-4ace-af4a-2b13453d4f8f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:27.427996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f8c17-bd05-4ace-af4a-2b13453d4f8f None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:27.429408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 105/209] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:27.435525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22e34a-faac-4784-9dfb-5aae3552745c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:27.441751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22e34a-faac-4784-9dfb-5aae3552745c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:27.441751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22e34a-faac-4784-9dfb-5aae3552745c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:27.455357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ee22e34a-faac-4784-9dfb-5aae3552745c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:27.456571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ee22e34a-faac-4784-9dfb-5aae3552745c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:27.524717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 105/210] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:27.546057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908f02b-bbad-4e05-bdbd-5c68b2a3b99a None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:27.546378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908f02b-bbad-4e05-bdbd-5c68b2a3b99a None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:27.546613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908f02b-bbad-4e05-bdbd-5c68b2a3b99a None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:27.547984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 106/211] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:27.553481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190aae6-f925-4ee0-ba93-bfbb20fae614 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:27.553805 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190aae6-f925-4ee0-ba93-bfbb20fae614 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:27.554094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190aae6-f925-4ee0-ba93-bfbb20fae614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:27.572827 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a190aae6-f925-4ee0-ba93-bfbb20fae614 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:27.574495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a190aae6-f925-4ee0-ba93-bfbb20fae614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:27.644912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 106/212] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:27.803252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908f02b-bbad-4e05-bdbd-5c68b2a3b99a None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:27.871910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:27.907151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908f02b-bbad-4e05-bdbd-5c68b2a3b99a None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:27.916912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190aae6-f925-4ee0-ba93-bfbb20fae614 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:27.934885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:27.948091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdb89f-d47e-48b5-bf65-dcb471f48784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:27.951601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdb89f-d47e-48b5-bf65-dcb471f48784 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:27.951601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdb89f-d47e-48b5-bf65-dcb471f48784 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:27.951601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdb89f-d47e-48b5-bf65-dcb471f48784 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:27.964267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdb89f-d47e-48b5-bf65-dcb471f48784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-2bbdb89f-d47e-48b5-bf65-dcb471f48784 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:27.975488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:27.976407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:27.977216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:27.980456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 107/213] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:27.994748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdb89f-d47e-48b5-bf65-dcb471f48784 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:28.036337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190aae6-f925-4ee0-ba93-bfbb20fae614 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:28.077894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11247974-a63e-4ed3-8371-45ad4e95ba47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:28.078596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247974-a63e-4ed3-8371-45ad4e95ba47 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:28.078884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247974-a63e-4ed3-8371-45ad4e95ba47 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:28.079161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247974-a63e-4ed3-8371-45ad4e95ba47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:28.080283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11247974-a63e-4ed3-8371-45ad4e95ba47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:28.081746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11247974-a63e-4ed3-8371-45ad4e95ba47 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:28.083326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 107/214] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:28.088932 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11247974-a63e-4ed3-8371-45ad4e95ba47 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:28.116134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdb89f-d47e-48b5-bf65-dcb471f48784 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:28.145612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:28.154827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34c55b-c31f-433d-85a7-451178d94030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:28.155438 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34c55b-c31f-433d-85a7-451178d94030 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:28.155721 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34c55b-c31f-433d-85a7-451178d94030 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:28.155997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34c55b-c31f-433d-85a7-451178d94030 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:28.156730 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34c55b-c31f-433d-85a7-451178d94030 None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:28.157043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34c55b-c31f-433d-85a7-451178d94030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:28.158710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34c55b-c31f-433d-85a7-451178d94030 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:28.168854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 108/215] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:28.175152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34c55b-c31f-433d-85a7-451178d94030 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:28.184431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:28.208972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11247974-a63e-4ed3-8371-45ad4e95ba47 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:28.250665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcb8d6-d2c5-43eb-8329-67dbff2748c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:28.251778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcb8d6-d2c5-43eb-8329-67dbff2748c3 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:28.252614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcb8d6-d2c5-43eb-8329-67dbff2748c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:28.253331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcb8d6-d2c5-43eb-8329-67dbff2748c3 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:28.254182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcb8d6-d2c5-43eb-8329-67dbff2748c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:28.256656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcb8d6-d2c5-43eb-8329-67dbff2748c3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:28.299139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34c55b-c31f-433d-85a7-451178d94030 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:28.337155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e65c0e-a74a-457f-98a7-199e85e003d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:28.337808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e65c0e-a74a-457f-98a7-199e85e003d2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:28.338091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e65c0e-a74a-457f-98a7-199e85e003d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:28.338363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e65c0e-a74a-457f-98a7-199e85e003d2 None admin] PATH_INFO: `/v3/users/9fb32af3d8d5443eab6b1404d07a9743` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:28.353691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e65c0e-a74a-457f-98a7-199e85e003d2 None admin] RBAC: Authorizing `identity:get_user(user_id=9fb32af3d8d5443eab6b1404d07a9743)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:28.355299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e65c0e-a74a-457f-98a7-199e85e003d2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:28.357207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 109/216] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:28 2022] GET /identity/v3/users/9fb32af3d8d5443eab6b1404d07a9743 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:28.362574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e65c0e-a74a-457f-98a7-199e85e003d2 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:28.378638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 108/217] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:29:28 2022] POST /identity/v3/users => generated 297 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:29:28.424435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:28.485222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e65c0e-a74a-457f-98a7-199e85e003d2 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:28.528489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c852ff9-815e-4f3c-8a72-bb435f2311a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:28.529114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c852ff9-815e-4f3c-8a72-bb435f2311a2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:28.529395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c852ff9-815e-4f3c-8a72-bb435f2311a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:28.529669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c852ff9-815e-4f3c-8a72-bb435f2311a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:28.606404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c852ff9-815e-4f3c-8a72-bb435f2311a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:28.607766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c852ff9-815e-4f3c-8a72-bb435f2311a2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:28.614639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b7712-6898-4803-91d2-528572ce1079 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:28.614639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b7712-6898-4803-91d2-528572ce1079 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:28.614639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b7712-6898-4803-91d2-528572ce1079 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:28.615229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 109/218] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:28.620756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15089147-bdf5-41be-87a6-a10d65306b3c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:28.621124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15089147-bdf5-41be-87a6-a10d65306b3c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:28.621403 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15089147-bdf5-41be-87a6-a10d65306b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:28.647021 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-15089147-bdf5-41be-87a6-a10d65306b3c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:28.648205 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-15089147-bdf5-41be-87a6-a10d65306b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:28.669898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:28.734094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 110/219] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 03:29:28.740071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c852ff9-815e-4f3c-8a72-bb435f2311a2 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:28.740663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 110/220] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:28.864690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c852ff9-815e-4f3c-8a72-bb435f2311a2 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:28.893936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a3cb9-29c9-4303-8aa5-84e1d30afc0c None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:28.894216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a3cb9-29c9-4303-8aa5-84e1d30afc0c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:28.894491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a3cb9-29c9-4303-8aa5-84e1d30afc0c None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:28.896173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 111/221] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:29:28.903567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122de75-ea92-4c5b-8516-751c2f49028c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:28.903978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122de75-ea92-4c5b-8516-751c2f49028c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:28.904253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122de75-ea92-4c5b-8516-751c2f49028c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:28.908191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44166e77-6fcf-4aa3-9e34-ae00a291719f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:28.908834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44166e77-6fcf-4aa3-9e34-ae00a291719f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:28.909113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44166e77-6fcf-4aa3-9e34-ae00a291719f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:28.909388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44166e77-6fcf-4aa3-9e34-ae00a291719f None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:28.910669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44166e77-6fcf-4aa3-9e34-ae00a291719f None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:28.912127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44166e77-6fcf-4aa3-9e34-ae00a291719f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:28.913821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 111/222] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:28 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:28.920811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44166e77-6fcf-4aa3-9e34-ae00a291719f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:28.926163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1122de75-ea92-4c5b-8516-751c2f49028c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:28.927389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1122de75-ea92-4c5b-8516-751c2f49028c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:28.948902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:29.020921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:29.028650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 112/223] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:29.042467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122de75-ea92-4c5b-8516-751c2f49028c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:29.059251 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44166e77-6fcf-4aa3-9e34-ae00a291719f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:29.130546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865536c4-f3e8-4124-9057-8c01030afcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:29.131086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865536c4-f3e8-4124-9057-8c01030afcdb None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:29.131358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865536c4-f3e8-4124-9057-8c01030afcdb None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:29.131626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865536c4-f3e8-4124-9057-8c01030afcdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:29.132366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865536c4-f3e8-4124-9057-8c01030afcdb None admin] RBAC: Adding query filter params (role.id=d801ebea910445a7a6100b35e07630dd, scope.project.id=e02b0fbedb42442bab7a31b8db209acb, user.id=9fb32af3d8d5443eab6b1404d07a9743) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:29.132669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865536c4-f3e8-4124-9057-8c01030afcdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:29.134197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865536c4-f3e8-4124-9057-8c01030afcdb None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:29.153577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122de75-ea92-4c5b-8516-751c2f49028c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:29.155847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 112/224] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:28 2022] GET /identity/v3/role_assignments?user.id=9fb32af3d8d5443eab6b1404d07a9743&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=d801ebea910445a7a6100b35e07630dd => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:29.205648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b3be-47e8-4933-bce8-a71390f8d76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:29.206349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b3be-47e8-4933-bce8-a71390f8d76d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:29.206632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b3be-47e8-4933-bce8-a71390f8d76d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:29.206906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b3be-47e8-4933-bce8-a71390f8d76d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:29.219639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2b3be-47e8-4933-bce8-a71390f8d76d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-0eb2b3be-47e8-4933-bce8-a71390f8d76d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:29.224541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:29.225455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:29.226319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 113/225] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:29:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:29.232411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865536c4-f3e8-4124-9057-8c01030afcdb None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:29.354071 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b3be-47e8-4933-bce8-a71390f8d76d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:29.385427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:29.425213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865536c4-f3e8-4124-9057-8c01030afcdb None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:29.468651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:29.483997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4447eb-12a1-4804-8aa9-17d87eb8da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:29.485408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4447eb-12a1-4804-8aa9-17d87eb8da05 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:29.486009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4447eb-12a1-4804-8aa9-17d87eb8da05 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:29.486009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4447eb-12a1-4804-8aa9-17d87eb8da05 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:29.486653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4447eb-12a1-4804-8aa9-17d87eb8da05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:29.486952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4447eb-12a1-4804-8aa9-17d87eb8da05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:29.488377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4447eb-12a1-4804-8aa9-17d87eb8da05 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:29.501758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 113/226] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:29.504406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4447eb-12a1-4804-8aa9-17d87eb8da05 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:29.577146 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b3be-47e8-4933-bce8-a71390f8d76d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:29.619000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4447eb-12a1-4804-8aa9-17d87eb8da05 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:29.663840 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237df06-7480-4664-a295-cd1062066bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:29.664776 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237df06-7480-4664-a295-cd1062066bad None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:29.665193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237df06-7480-4664-a295-cd1062066bad None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:29.665614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237df06-7480-4664-a295-cd1062066bad None admin] PATH_INFO: `/v3/users/660d48ed2ad942859b0cf43abaa925aa` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:29.668087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccc28d-93f1-41cb-bbc7-fa721d1c6c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:29.668813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccc28d-93f1-41cb-bbc7-fa721d1c6c61 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:29.669090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccc28d-93f1-41cb-bbc7-fa721d1c6c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:29.669358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccc28d-93f1-41cb-bbc7-fa721d1c6c61 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:29.670486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237df06-7480-4664-a295-cd1062066bad None admin] RBAC: Authorizing `identity:get_user(user_id=660d48ed2ad942859b0cf43abaa925aa)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:29.673162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237df06-7480-4664-a295-cd1062066bad None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:29.674792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 114/227] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:29 2022] GET /identity/v3/users/660d48ed2ad942859b0cf43abaa925aa => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:29.680471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237df06-7480-4664-a295-cd1062066bad None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:29.696398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccc28d-93f1-41cb-bbc7-fa721d1c6c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e5ccc28d-93f1-41cb-bbc7-fa721d1c6c61 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:29.706505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:29.708159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:29.709712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:29.710945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 114/228] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:29.722961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccc28d-93f1-41cb-bbc7-fa721d1c6c61 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:29.726091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:29.909921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccc28d-93f1-41cb-bbc7-fa721d1c6c61 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:29.916516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237df06-7480-4664-a295-cd1062066bad None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:29.932775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:29.950455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2991c61-3430-456d-8840-32ff53e4afb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:29.951125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2991c61-3430-456d-8840-32ff53e4afb3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:29.951399 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2991c61-3430-456d-8840-32ff53e4afb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:29.951665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2991c61-3430-456d-8840-32ff53e4afb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:29.952373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2991c61-3430-456d-8840-32ff53e4afb3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:29.952675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2991c61-3430-456d-8840-32ff53e4afb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:29.954258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2991c61-3430-456d-8840-32ff53e4afb3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:29.963059 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 115/229] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:29.970666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2991c61-3430-456d-8840-32ff53e4afb3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:30.007379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615c3b5-145c-456d-b02d-7344e0a408a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:30.008032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615c3b5-145c-456d-b02d-7344e0a408a1 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:30.008319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615c3b5-145c-456d-b02d-7344e0a408a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:30.008597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615c3b5-145c-456d-b02d-7344e0a408a1 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:30.013770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615c3b5-145c-456d-b02d-7344e0a408a1 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:30.015307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615c3b5-145c-456d-b02d-7344e0a408a1 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:30.017309 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 115/230] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:29 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:30.022782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615c3b5-145c-456d-b02d-7344e0a408a1 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:30.093683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2991c61-3430-456d-8840-32ff53e4afb3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:30.133177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db702a-4dd8-4187-86a8-806f846d3411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:30.133983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db702a-4dd8-4187-86a8-806f846d3411 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:30.134277 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db702a-4dd8-4187-86a8-806f846d3411 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:30.134549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db702a-4dd8-4187-86a8-806f846d3411 None admin] PATH_INFO: `/v3/users/660d48ed2ad942859b0cf43abaa925aa` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:30.135682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db702a-4dd8-4187-86a8-806f846d3411 None admin] RBAC: Authorizing `identity:get_user(user_id=660d48ed2ad942859b0cf43abaa925aa)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:30.137335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db702a-4dd8-4187-86a8-806f846d3411 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:30.139184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 116/231] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:29 2022] GET /identity/v3/users/660d48ed2ad942859b0cf43abaa925aa => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:30.144644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db702a-4dd8-4187-86a8-806f846d3411 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:30.200811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:30.257919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db702a-4dd8-4187-86a8-806f846d3411 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:30.259616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:30.298058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7854cd3-6dec-4a88-a36e-729e67316786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:30.298692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7854cd3-6dec-4a88-a36e-729e67316786 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:30.298972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7854cd3-6dec-4a88-a36e-729e67316786 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:30.299247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7854cd3-6dec-4a88-a36e-729e67316786 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:30.300691 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7854cd3-6dec-4a88-a36e-729e67316786 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:30.302293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7854cd3-6dec-4a88-a36e-729e67316786 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:30.304118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 117/232] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:30 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:30.306248 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615c3b5-145c-456d-b02d-7344e0a408a1 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:30.311196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7854cd3-6dec-4a88-a36e-729e67316786 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:30.419145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393f7b4-2d32-40e2-b8d3-b3887e6f20c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:30.419825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393f7b4-2d32-40e2-b8d3-b3887e6f20c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:30.420163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393f7b4-2d32-40e2-b8d3-b3887e6f20c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:30.420444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393f7b4-2d32-40e2-b8d3-b3887e6f20c6 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb/users/660d48ed2ad942859b0cf43abaa925aa/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:30.422439 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393f7b4-2d32-40e2-b8d3-b3887e6f20c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e02b0fbedb42442bab7a31b8db209acb, user_id=660d48ed2ad942859b0cf43abaa925aa, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:30.423898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393f7b4-2d32-40e2-b8d3-b3887e6f20c6 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:30.438325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7854cd3-6dec-4a88-a36e-729e67316786 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:30.457244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:30.475167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38562221-c062-4788-b8bc-7f53ebd15334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:30.475821 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38562221-c062-4788-b8bc-7f53ebd15334 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:30.476099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38562221-c062-4788-b8bc-7f53ebd15334 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:30.476597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38562221-c062-4788-b8bc-7f53ebd15334 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb/users/660d48ed2ad942859b0cf43abaa925aa/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:30.478815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38562221-c062-4788-b8bc-7f53ebd15334 None admin] RBAC: Authorizing `identity:create_grant(project_id=e02b0fbedb42442bab7a31b8db209acb, user_id=660d48ed2ad942859b0cf43abaa925aa, role_id=ec3ad2e5f7314c3b8d35826cfd81b5ba)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:30.480996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38562221-c062-4788-b8bc-7f53ebd15334 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:30.533656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 116/233] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:29:30 2022] PUT /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb/users/660d48ed2ad942859b0cf43abaa925aa/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:29:30.581077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 118/234] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:29:30 2022] PUT /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb/users/660d48ed2ad942859b0cf43abaa925aa/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:29:31.020187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c55ecb-13a5-4ae9-ab65-88bf9f2d57a9 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:31.020502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c55ecb-13a5-4ae9-ab65-88bf9f2d57a9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:31.020749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c55ecb-13a5-4ae9-ab65-88bf9f2d57a9 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:31.022450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 117/235] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:31.028610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f1cf8-f86b-4345-a4cc-1c4b4a2de78c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:31.029049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f1cf8-f86b-4345-a4cc-1c4b4a2de78c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:31.029227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f1cf8-f86b-4345-a4cc-1c4b4a2de78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:31.048304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a34f1cf8-f86b-4345-a4cc-1c4b4a2de78c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:31.049505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a34f1cf8-f86b-4345-a4cc-1c4b4a2de78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:31.121922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 119/236] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:31.261854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998847b-24cc-4a6a-b06c-7f0bc275fd8b None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:31.262173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998847b-24cc-4a6a-b06c-7f0bc275fd8b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:31.262414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998847b-24cc-4a6a-b06c-7f0bc275fd8b None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:31.264064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 118/237] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:31.269375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413f5e7-f32b-421e-9309-47325fdfde5b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:31.269666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413f5e7-f32b-421e-9309-47325fdfde5b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:31.269977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413f5e7-f32b-421e-9309-47325fdfde5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:31.295100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9413f5e7-f32b-421e-9309-47325fdfde5b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:31.296599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9413f5e7-f32b-421e-9309-47325fdfde5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:31.384337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 120/238] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:31.397572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998847b-24cc-4a6a-b06c-7f0bc275fd8b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:31.534514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998847b-24cc-4a6a-b06c-7f0bc275fd8b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:31.571836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2351e-fff3-4ca3-9635-ed11c0dd9688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:31.572468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2351e-fff3-4ca3-9635-ed11c0dd9688 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:31.572741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2351e-fff3-4ca3-9635-ed11c0dd9688 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:31.573005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2351e-fff3-4ca3-9635-ed11c0dd9688 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:31.580731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:31.585179 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2351e-fff3-4ca3-9635-ed11c0dd9688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-2fb2351e-fff3-4ca3-9635-ed11c0dd9688 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:31.590502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:31.592422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:31.593229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 119/239] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:31.602823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413f5e7-f32b-421e-9309-47325fdfde5b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:31.669939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:31.680889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2351e-fff3-4ca3-9635-ed11c0dd9688 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:31.794570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413f5e7-f32b-421e-9309-47325fdfde5b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:31.850652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2351e-fff3-4ca3-9635-ed11c0dd9688 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:31.860173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2ebad-0446-4157-95d7-609577d042ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:31.861159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2ebad-0446-4157-95d7-609577d042ad None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:31.861598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2ebad-0446-4157-95d7-609577d042ad None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:31.862038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2ebad-0446-4157-95d7-609577d042ad None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:31.862954 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2ebad-0446-4157-95d7-609577d042ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:31.863402 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2ebad-0446-4157-95d7-609577d042ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:31.864951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2ebad-0446-4157-95d7-609577d042ad None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:31.879675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 121/240] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:31.881363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb52e84-f319-4533-a96d-bf6983ca5ac2 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:31.881920 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb52e84-f319-4533-a96d-bf6983ca5ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:31.882335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb52e84-f319-4533-a96d-bf6983ca5ac2 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:31.884238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 122/241] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:31.889497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb52e84-f319-4533-a96d-bf6983ca5ac2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:31.891016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad189d3-4ccd-4f41-a251-dba2640e8eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:31.891875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad189d3-4ccd-4f41-a251-dba2640e8eef None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:31.892329 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad189d3-4ccd-4f41-a251-dba2640e8eef None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:31.892763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad189d3-4ccd-4f41-a251-dba2640e8eef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:31.894148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad189d3-4ccd-4f41-a251-dba2640e8eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:31.895804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad189d3-4ccd-4f41-a251-dba2640e8eef None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:31.897821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 120/242] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:31.899163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d0b38-0faa-4af8-afc3-dad59762a458 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:31.899565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d0b38-0faa-4af8-afc3-dad59762a458 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:31.899975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d0b38-0faa-4af8-afc3-dad59762a458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:31.918484 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-0c6d0b38-0faa-4af8-afc3-dad59762a458 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:31.919918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0c6d0b38-0faa-4af8-afc3-dad59762a458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:31.980717 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:31.988868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 121/243] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:31.993933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d0b38-0faa-4af8-afc3-dad59762a458 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:32.015497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:32.028702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb52e84-f319-4533-a96d-bf6983ca5ac2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:32.098754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed377c-3a8a-44a8-bb8c-985488e5e03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:32.099374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed377c-3a8a-44a8-bb8c-985488e5e03e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:32.099650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed377c-3a8a-44a8-bb8c-985488e5e03e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:32.099922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed377c-3a8a-44a8-bb8c-985488e5e03e None admin] PATH_INFO: `/v3/users/a75fef9313fe4fe9b5081ca64ef1ff73` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:32.114971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed377c-3a8a-44a8-bb8c-985488e5e03e None admin] RBAC: Authorizing `identity:get_user(user_id=a75fef9313fe4fe9b5081ca64ef1ff73)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:32.116478 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed377c-3a8a-44a8-bb8c-985488e5e03e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:32.118944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 123/244] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:31 2022] GET /identity/v3/users/a75fef9313fe4fe9b5081ca64ef1ff73 => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:32.125097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed377c-3a8a-44a8-bb8c-985488e5e03e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:32.196386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d0b38-0faa-4af8-afc3-dad59762a458 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:32.242288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8c885-f024-4b4e-9681-70ed70e7d2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:32.242811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8c885-f024-4b4e-9681-70ed70e7d2a6 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:32.243088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8c885-f024-4b4e-9681-70ed70e7d2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:32.243399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8c885-f024-4b4e-9681-70ed70e7d2a6 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:32.244046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8c885-f024-4b4e-9681-70ed70e7d2a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:32.246043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8c885-f024-4b4e-9681-70ed70e7d2a6 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:32.289646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed377c-3a8a-44a8-bb8c-985488e5e03e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:32.325477 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848026a-c540-45c4-82ed-b5b8d9a5e390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:32.326114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848026a-c540-45c4-82ed-b5b8d9a5e390 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:32.326395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848026a-c540-45c4-82ed-b5b8d9a5e390 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:32.326668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848026a-c540-45c4-82ed-b5b8d9a5e390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:32.335030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 122/245] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:29:31 2022] POST /identity/v3/users => generated 293 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:29:32.354425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8c885-f024-4b4e-9681-70ed70e7d2a6 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:32.404565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:32.462027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:32.486652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848026a-c540-45c4-82ed-b5b8d9a5e390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:32.488167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848026a-c540-45c4-82ed-b5b8d9a5e390 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:32.552657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8c885-f024-4b4e-9681-70ed70e7d2a6 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:32.600576 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c357bc-a1a4-4fb7-8589-d6974bcd2427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:32.601214 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c357bc-a1a4-4fb7-8589-d6974bcd2427 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:32.601487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c357bc-a1a4-4fb7-8589-d6974bcd2427 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:32.601949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c357bc-a1a4-4fb7-8589-d6974bcd2427 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:32.616438 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c357bc-a1a4-4fb7-8589-d6974bcd2427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-78c357bc-a1a4-4fb7-8589-d6974bcd2427 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:32.624180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:32.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:32.626387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:32.628508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 123/246] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:32.638714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c357bc-a1a4-4fb7-8589-d6974bcd2427 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:32.663055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 124/247] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:32.668852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848026a-c540-45c4-82ed-b5b8d9a5e390 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:32.741801 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:32.767423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c357bc-a1a4-4fb7-8589-d6974bcd2427 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:32.782995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:32.804677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848026a-c540-45c4-82ed-b5b8d9a5e390 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:32.843747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc507e-6c3d-4e6e-abf9-a8c8d831b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:32.844369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc507e-6c3d-4e6e-abf9-a8c8d831b93d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:32.844643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc507e-6c3d-4e6e-abf9-a8c8d831b93d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:32.844913 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc507e-6c3d-4e6e-abf9-a8c8d831b93d None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:32.845920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc507e-6c3d-4e6e-abf9-a8c8d831b93d None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:32.846221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc507e-6c3d-4e6e-abf9-a8c8d831b93d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:32.848270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc507e-6c3d-4e6e-abf9-a8c8d831b93d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:32.853038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ed349-43b9-40fc-a973-da3379db0d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:32.853967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ed349-43b9-40fc-a973-da3379db0d19 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:32.854456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ed349-43b9-40fc-a973-da3379db0d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:32.856285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ed349-43b9-40fc-a973-da3379db0d19 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:32.857787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ed349-43b9-40fc-a973-da3379db0d19 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:32.858843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 124/248] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:32.862419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ed349-43b9-40fc-a973-da3379db0d19 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:32.864598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 125/249] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:32 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:32.865630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc507e-6c3d-4e6e-abf9-a8c8d831b93d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:32.872773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ed349-43b9-40fc-a973-da3379db0d19 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:33.014217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc507e-6c3d-4e6e-abf9-a8c8d831b93d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:33.019298 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ed349-43b9-40fc-a973-da3379db0d19 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:33.053500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:33.064820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849beaa-b5d2-4569-9581-62a46c71b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:33.064820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849beaa-b5d2-4569-9581-62a46c71b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:33.064820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849beaa-b5d2-4569-9581-62a46c71b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:33.064820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849beaa-b5d2-4569-9581-62a46c71b6f2 None admin] PATH_INFO: `/v3/users/9fb32af3d8d5443eab6b1404d07a9743` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:33.065197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849beaa-b5d2-4569-9581-62a46c71b6f2 None admin] RBAC: Authorizing `identity:get_user(user_id=9fb32af3d8d5443eab6b1404d07a9743)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:33.066428 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8137c-d243-474b-979c-a27c69686941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:33.067240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8137c-d243-474b-979c-a27c69686941 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:33.067670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8137c-d243-474b-979c-a27c69686941 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:33.068090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8137c-d243-474b-979c-a27c69686941 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:33.068978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8137c-d243-474b-979c-a27c69686941 None admin] RBAC: Adding query filter params (role.id=ed414442e2d24f2f99e36d88ade1835e, scope.project.id=e02b0fbedb42442bab7a31b8db209acb, user.id=a75fef9313fe4fe9b5081ca64ef1ff73) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:33.069435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8137c-d243-474b-979c-a27c69686941 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:33.071076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8137c-d243-474b-979c-a27c69686941 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:33.072037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849beaa-b5d2-4569-9581-62a46c71b6f2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:33.078379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 125/250] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:32 2022] GET /identity/v3/users/9fb32af3d8d5443eab6b1404d07a9743 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:33.082839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849beaa-b5d2-4569-9581-62a46c71b6f2 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:33.083945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 126/251] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:32 2022] GET /identity/v3/role_assignments?user.id=a75fef9313fe4fe9b5081ca64ef1ff73&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=ed414442e2d24f2f99e36d88ade1835e => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:33.144304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:33.210759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849beaa-b5d2-4569-9581-62a46c71b6f2 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:33.256021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee7222-d90b-4265-84a1-df7cf02325f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:33.256728 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee7222-d90b-4265-84a1-df7cf02325f0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:33.257018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee7222-d90b-4265-84a1-df7cf02325f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:33.257296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee7222-d90b-4265-84a1-df7cf02325f0 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:33.262741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee7222-d90b-4265-84a1-df7cf02325f0 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:33.264265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee7222-d90b-4265-84a1-df7cf02325f0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:33.266332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 126/252] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:33 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:33.272505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8137c-d243-474b-979c-a27c69686941 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:33.389681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8137c-d243-474b-979c-a27c69686941 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:33.432994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cefd9-731d-49ea-a333-eec9a7ff45ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:33.442373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cefd9-731d-49ea-a333-eec9a7ff45ee None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:33.442373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cefd9-731d-49ea-a333-eec9a7ff45ee None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:33.442373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cefd9-731d-49ea-a333-eec9a7ff45ee None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb/users/9fb32af3d8d5443eab6b1404d07a9743/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:33.442373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cefd9-731d-49ea-a333-eec9a7ff45ee None admin] RBAC: Authorizing `identity:create_grant(project_id=e02b0fbedb42442bab7a31b8db209acb, user_id=9fb32af3d8d5443eab6b1404d07a9743, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:33.442373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cefd9-731d-49ea-a333-eec9a7ff45ee None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:33.491902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efc6c9-1480-4356-b66e-4213a51ee875 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:33.492216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efc6c9-1480-4356-b66e-4213a51ee875 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:33.492463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efc6c9-1480-4356-b66e-4213a51ee875 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:33.494365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 127/253] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:33.500180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3e6f-38fa-47b2-be50-17d4abfc5d15 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:33.500422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3e6f-38fa-47b2-be50-17d4abfc5d15 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:33.500762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3e6f-38fa-47b2-be50-17d4abfc5d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:33.524151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b29d3e6f-38fa-47b2-be50-17d4abfc5d15 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:33.530470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 127/254] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:29:33 2022] PUT /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb/users/9fb32af3d8d5443eab6b1404d07a9743/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:33.531972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7abd43-d8d7-4a8a-bf7e-f6c7911c5b08 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:33.534109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7abd43-d8d7-4a8a-bf7e-f6c7911c5b08 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:33.534109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7abd43-d8d7-4a8a-bf7e-f6c7911c5b08 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:33.534275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b29d3e6f-38fa-47b2-be50-17d4abfc5d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:33.535570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 128/255] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:33.543471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347d496-0a77-4957-81a3-95686dccac76 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:33.543841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347d496-0a77-4957-81a3-95686dccac76 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:33.544074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347d496-0a77-4957-81a3-95686dccac76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:33.579875 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5347d496-0a77-4957-81a3-95686dccac76 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:33.581100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5347d496-0a77-4957-81a3-95686dccac76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:33.673290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 128/256] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:33.733478 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 129/257] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:33.733478 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:34.033351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d3e6f-38fa-47b2-be50-17d4abfc5d15 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:34.082879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347d496-0a77-4957-81a3-95686dccac76 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:34.087055 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:34.154368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d3e6f-38fa-47b2-be50-17d4abfc5d15 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:34.190975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395950a1-6074-4385-a258-a83471c4a59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:34.191708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395950a1-6074-4385-a258-a83471c4a59a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:34.191985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395950a1-6074-4385-a258-a83471c4a59a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:34.192252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395950a1-6074-4385-a258-a83471c4a59a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:34.203497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395950a1-6074-4385-a258-a83471c4a59a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-395950a1-6074-4385-a258-a83471c4a59a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:34.208777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:34.209740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:34.210592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 129/258] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:34.216512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395950a1-6074-4385-a258-a83471c4a59a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:34.254212 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347d496-0a77-4957-81a3-95686dccac76 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:34.319933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee13c1c-d796-4db3-861e-ba0c8e35ea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:34.320546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee13c1c-d796-4db3-861e-ba0c8e35ea58 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:34.320839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee13c1c-d796-4db3-861e-ba0c8e35ea58 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:34.321118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee13c1c-d796-4db3-861e-ba0c8e35ea58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:34.341287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee13c1c-d796-4db3-861e-ba0c8e35ea58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-9ee13c1c-d796-4db3-861e-ba0c8e35ea58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:34.354732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:34.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:34.358750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 130/259] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:29:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:34.363443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee13c1c-d796-4db3-861e-ba0c8e35ea58 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:34.375840 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:34.384454 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395950a1-6074-4385-a258-a83471c4a59a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:34.446969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d6f5f-4cc3-4898-b7e7-b0940069c56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:34.447726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d6f5f-4cc3-4898-b7e7-b0940069c56b None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:34.448007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d6f5f-4cc3-4898-b7e7-b0940069c56b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:34.448746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d6f5f-4cc3-4898-b7e7-b0940069c56b None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:34.449511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d6f5f-4cc3-4898-b7e7-b0940069c56b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:34.449832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d6f5f-4cc3-4898-b7e7-b0940069c56b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:34.451226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d6f5f-4cc3-4898-b7e7-b0940069c56b None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:34.469377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 130/260] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:34.470768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d6f5f-4cc3-4898-b7e7-b0940069c56b None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:34.474053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:34.595077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee13c1c-d796-4db3-861e-ba0c8e35ea58 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:34.644913 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d6f5f-4cc3-4898-b7e7-b0940069c56b None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:34.652103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0724c3-1766-4ca3-a016-55576c68105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:34.652724 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0724c3-1766-4ca3-a016-55576c68105e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:34.653008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0724c3-1766-4ca3-a016-55576c68105e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:34.653286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0724c3-1766-4ca3-a016-55576c68105e None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:34.653987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0724c3-1766-4ca3-a016-55576c68105e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:34.654292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0724c3-1766-4ca3-a016-55576c68105e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:34.655787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0724c3-1766-4ca3-a016-55576c68105e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:34.670165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 131/261] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:34.675929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0724c3-1766-4ca3-a016-55576c68105e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:34.713437 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c0c94-9d64-436b-9d38-ccbc73ed4c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:34.714250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c0c94-9d64-436b-9d38-ccbc73ed4c6d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:34.714657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c0c94-9d64-436b-9d38-ccbc73ed4c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:34.714945 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c0c94-9d64-436b-9d38-ccbc73ed4c6d None admin] PATH_INFO: `/v3/users/660d48ed2ad942859b0cf43abaa925aa` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:34.718052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c0c94-9d64-436b-9d38-ccbc73ed4c6d None admin] RBAC: Authorizing `identity:get_user(user_id=660d48ed2ad942859b0cf43abaa925aa)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:34.719591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c0c94-9d64-436b-9d38-ccbc73ed4c6d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:34.721512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 131/262] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:34 2022] GET /identity/v3/users/660d48ed2ad942859b0cf43abaa925aa => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:34.726811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c0c94-9d64-436b-9d38-ccbc73ed4c6d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:34.781742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:34.840799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0724c3-1766-4ca3-a016-55576c68105e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:34.848707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:34.913484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34da7d3-3ce3-4a98-a014-13c4d8e7c8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:34.915780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34da7d3-3ce3-4a98-a014-13c4d8e7c8da None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:34.918157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34da7d3-3ce3-4a98-a014-13c4d8e7c8da None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:34.918617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34da7d3-3ce3-4a98-a014-13c4d8e7c8da None admin] PATH_INFO: `/v3/users/660d48ed2ad942859b0cf43abaa925aa` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:34.920036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34da7d3-3ce3-4a98-a014-13c4d8e7c8da None admin] RBAC: Authorizing `identity:get_user(user_id=660d48ed2ad942859b0cf43abaa925aa)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:34.922615 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34da7d3-3ce3-4a98-a014-13c4d8e7c8da None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:34.927571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 132/263] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:34 2022] GET /identity/v3/users/660d48ed2ad942859b0cf43abaa925aa => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:34.945252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c0c94-9d64-436b-9d38-ccbc73ed4c6d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:34.951541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34da7d3-3ce3-4a98-a014-13c4d8e7c8da None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:35.005837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6731422-2aba-4032-8118-bdca2a918518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:35.006734 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6731422-2aba-4032-8118-bdca2a918518 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:35.007444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6731422-2aba-4032-8118-bdca2a918518 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:35.007881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6731422-2aba-4032-8118-bdca2a918518 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:35.091928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6731422-2aba-4032-8118-bdca2a918518 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:35.096342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6731422-2aba-4032-8118-bdca2a918518 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:35.130477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:35.187798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34da7d3-3ce3-4a98-a014-13c4d8e7c8da None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:35.241553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 132/264] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 03:29:35.249568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6731422-2aba-4032-8118-bdca2a918518 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:35.258630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12af84-050f-487d-b76e-a3e3ccdfd0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:35.259451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12af84-050f-487d-b76e-a3e3ccdfd0fe None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:35.259891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12af84-050f-487d-b76e-a3e3ccdfd0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:35.260320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12af84-050f-487d-b76e-a3e3ccdfd0fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:35.331601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:35.372778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6731422-2aba-4032-8118-bdca2a918518 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:35.380570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12af84-050f-487d-b76e-a3e3ccdfd0fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:35.382280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12af84-050f-487d-b76e-a3e3ccdfd0fe None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:35.402693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:35.408829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab059207-455f-4810-979c-d83a303353df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:35.409689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab059207-455f-4810-979c-d83a303353df None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:35.410181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab059207-455f-4810-979c-d83a303353df None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:35.410637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab059207-455f-4810-979c-d83a303353df None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:35.411937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab059207-455f-4810-979c-d83a303353df None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:35.413635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab059207-455f-4810-979c-d83a303353df None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:35.420164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 133/265] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:35 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:35.423321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab059207-455f-4810-979c-d83a303353df None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:35.541256 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab059207-455f-4810-979c-d83a303353df None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:35.582022 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782aa214-e80e-41ea-ac79-4fcbe6475c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:35.582946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782aa214-e80e-41ea-ac79-4fcbe6475c12 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:35.583406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782aa214-e80e-41ea-ac79-4fcbe6475c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:35.583855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782aa214-e80e-41ea-ac79-4fcbe6475c12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:35.584761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782aa214-e80e-41ea-ac79-4fcbe6475c12 None admin] RBAC: Adding query filter params (role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba, scope.project.id=e02b0fbedb42442bab7a31b8db209acb, user.id=660d48ed2ad942859b0cf43abaa925aa) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:35.585268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782aa214-e80e-41ea-ac79-4fcbe6475c12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:35.587687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782aa214-e80e-41ea-ac79-4fcbe6475c12 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:35.592717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 133/266] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 03:29:35.597219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 134/267] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:35 2022] GET /identity/v3/role_assignments?user.id=660d48ed2ad942859b0cf43abaa925aa&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:35.598377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12af84-050f-487d-b76e-a3e3ccdfd0fe None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:35.691694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:35.730674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12af84-050f-487d-b76e-a3e3ccdfd0fe None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:35.774530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76780e-912b-4a0d-acfe-77473fdf087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:35.775165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76780e-912b-4a0d-acfe-77473fdf087e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:35.775507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76780e-912b-4a0d-acfe-77473fdf087e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:35.775787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76780e-912b-4a0d-acfe-77473fdf087e None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:35.778495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76780e-912b-4a0d-acfe-77473fdf087e None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:35.780245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76780e-912b-4a0d-acfe-77473fdf087e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:35.782738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 134/268] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:35 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:35.788388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782aa214-e80e-41ea-ac79-4fcbe6475c12 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:35.861480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:35.928550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782aa214-e80e-41ea-ac79-4fcbe6475c12 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:35.970230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023e859-b953-403c-868a-a63433e6b891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:35.970764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023e859-b953-403c-868a-a63433e6b891 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:35.971045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023e859-b953-403c-868a-a63433e6b891 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:35.971320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023e859-b953-403c-868a-a63433e6b891 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:35.972048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023e859-b953-403c-868a-a63433e6b891 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=e02b0fbedb42442bab7a31b8db209acb, user.id=660d48ed2ad942859b0cf43abaa925aa) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:35.972359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023e859-b953-403c-868a-a63433e6b891 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:35.973821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023e859-b953-403c-868a-a63433e6b891 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:35.982114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 135/269] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:35 2022] GET /identity/v3/role_assignments?user.id=660d48ed2ad942859b0cf43abaa925aa&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:36.239611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6323872-2285-43ce-b32e-7fbf6b68aa50 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:36.240090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6323872-2285-43ce-b32e-7fbf6b68aa50 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:36.240090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6323872-2285-43ce-b32e-7fbf6b68aa50 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:36.241692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 135/270] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:36.246988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f99aa-3b66-4d28-9144-1375828cad9c None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:36.247535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f99aa-3b66-4d28-9144-1375828cad9c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:36.248018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f99aa-3b66-4d28-9144-1375828cad9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:36.263415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4e40e-7cda-4864-a60c-f535acb6b18f None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:36.263942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4e40e-7cda-4864-a60c-f535acb6b18f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:36.264450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4e40e-7cda-4864-a60c-f535acb6b18f None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:36.266063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 136/271] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:29:36.272106 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02140e-fa69-472f-87d2-23db7bd219a4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:36.272638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b78f99aa-3b66-4d28-9144-1375828cad9c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:36.272857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02140e-fa69-472f-87d2-23db7bd219a4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:36.273352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02140e-fa69-472f-87d2-23db7bd219a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:36.273898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b78f99aa-3b66-4d28-9144-1375828cad9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:36.302039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5f02140e-fa69-472f-87d2-23db7bd219a4 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:36.303538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5f02140e-fa69-472f-87d2-23db7bd219a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:36.382752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 136/272] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:36.385199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 137/273] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:36.533783 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a988f3-184c-4a13-968a-f5a5ba4a202a None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:36.534269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a988f3-184c-4a13-968a-f5a5ba4a202a None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:36.534684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a988f3-184c-4a13-968a-f5a5ba4a202a None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:36.536518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 137/274] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:36.541946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db22d5-c961-4664-b371-0d27fad62778 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:36.542578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db22d5-c961-4664-b371-0d27fad62778 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:36.543050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db22d5-c961-4664-b371-0d27fad62778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:36.564560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c7db22d5-c961-4664-b371-0d27fad62778 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:36.566097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c7db22d5-c961-4664-b371-0d27fad62778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:36.633758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:36.639672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 138/275] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:36.667879 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a988f3-184c-4a13-968a-f5a5ba4a202a None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:36.683370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db22d5-c961-4664-b371-0d27fad62778 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:36.732652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:36.781188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a988f3-184c-4a13-968a-f5a5ba4a202a None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:36.824748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ff573-6f29-459c-aa29-5db11647c58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:36.826170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ff573-6f29-459c-aa29-5db11647c58e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:36.826642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ff573-6f29-459c-aa29-5db11647c58e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:36.827100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ff573-6f29-459c-aa29-5db11647c58e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:36.829275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db22d5-c961-4664-b371-0d27fad62778 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:36.838215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ff573-6f29-459c-aa29-5db11647c58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-315ff573-6f29-459c-aa29-5db11647c58e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:36.843518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:36.844413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:36.845289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:36.850905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 138/276] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:36.850905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97f2f-c124-4af7-b656-0035394ae6ab None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:36.850905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97f2f-c124-4af7-b656-0035394ae6ab None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:36.850905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97f2f-c124-4af7-b656-0035394ae6ab None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:36.853987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 139/277] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:36.855653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c97f2f-c124-4af7-b656-0035394ae6ab None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:36.871732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a83ba0-e603-4412-8ed4-da512942f8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:36.872725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a83ba0-e603-4412-8ed4-da512942f8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:36.873004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a83ba0-e603-4412-8ed4-da512942f8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:36.873274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a83ba0-e603-4412-8ed4-da512942f8e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:36.883736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a83ba0-e603-4412-8ed4-da512942f8e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-b7a83ba0-e603-4412-8ed4-da512942f8e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:36.889660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:36.890578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 139/278] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:36.896022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b291224-2d15-41f9-885b-22edc0579226 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:36.896750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b291224-2d15-41f9-885b-22edc0579226 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:36.896750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b291224-2d15-41f9-885b-22edc0579226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:36.914202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3b291224-2d15-41f9-885b-22edc0579226 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:36.915697 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3b291224-2d15-41f9-885b-22edc0579226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:36.969068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c97f2f-c124-4af7-b656-0035394ae6ab None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:36.973161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:36.985585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:36.986935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 140/279] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:36.987670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b291224-2d15-41f9-885b-22edc0579226 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.004424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe032d3e-2907-4655-839b-1327d2e83e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.005025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe032d3e-2907-4655-839b-1327d2e83e9c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.005300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe032d3e-2907-4655-839b-1327d2e83e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.005626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe032d3e-2907-4655-839b-1327d2e83e9c None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.006432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe032d3e-2907-4655-839b-1327d2e83e9c None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:37.006733 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe032d3e-2907-4655-839b-1327d2e83e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:37.008144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe032d3e-2907-4655-839b-1327d2e83e9c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:37.017338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 140/280] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:37.017338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe032d3e-2907-4655-839b-1327d2e83e9c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.107731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b291224-2d15-41f9-885b-22edc0579226 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:37.130296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe032d3e-2907-4655-839b-1327d2e83e9c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:37.142289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1056fa-bb13-4095-8928-042e33c92942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.142893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1056fa-bb13-4095-8928-042e33c92942 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.143171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1056fa-bb13-4095-8928-042e33c92942 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.143447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1056fa-bb13-4095-8928-042e33c92942 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.144205 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1056fa-bb13-4095-8928-042e33c92942 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:37.144502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1056fa-bb13-4095-8928-042e33c92942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:37.146011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1056fa-bb13-4095-8928-042e33c92942 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:37.154680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 141/281] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:37.155370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1056fa-bb13-4095-8928-042e33c92942 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.171199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5561e88-78c2-4c64-b772-301fa459b853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.171827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5561e88-78c2-4c64-b772-301fa459b853 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.172109 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5561e88-78c2-4c64-b772-301fa459b853 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.172611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5561e88-78c2-4c64-b772-301fa459b853 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.173492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5561e88-78c2-4c64-b772-301fa459b853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:37.174960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5561e88-78c2-4c64-b772-301fa459b853 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:37.176692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 141/282] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:37.177294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5561e88-78c2-4c64-b772-301fa459b853 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.208990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:37.232392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:37.258322 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1056fa-bb13-4095-8928-042e33c92942 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:37.289011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5561e88-78c2-4c64-b772-301fa459b853 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:37.291778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f94b5-3b5b-4ae7-92d4-c79606e4f08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.292635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f94b5-3b5b-4ae7-92d4-c79606e4f08e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.292635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f94b5-3b5b-4ae7-92d4-c79606e4f08e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.292962 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f94b5-3b5b-4ae7-92d4-c79606e4f08e None admin] PATH_INFO: `/v3/users/2a8d693f7cf549e5bbe6558fd90f3fae` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.307002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f94b5-3b5b-4ae7-92d4-c79606e4f08e None admin] RBAC: Authorizing `identity:get_user(user_id=2a8d693f7cf549e5bbe6558fd90f3fae)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:37.309129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f94b5-3b5b-4ae7-92d4-c79606e4f08e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:37.315381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 142/283] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:37 2022] GET /identity/v3/users/2a8d693f7cf549e5bbe6558fd90f3fae => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:37.315381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f94b5-3b5b-4ae7-92d4-c79606e4f08e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.324301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4d5d5-9e17-402f-8fc8-13996f26e857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.324931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4d5d5-9e17-402f-8fc8-13996f26e857 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.325210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4d5d5-9e17-402f-8fc8-13996f26e857 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.325480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4d5d5-9e17-402f-8fc8-13996f26e857 None admin] PATH_INFO: `/v3/users/a75fef9313fe4fe9b5081ca64ef1ff73` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.326633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4d5d5-9e17-402f-8fc8-13996f26e857 None admin] RBAC: Authorizing `identity:get_user(user_id=a75fef9313fe4fe9b5081ca64ef1ff73)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:37.328062 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4d5d5-9e17-402f-8fc8-13996f26e857 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:37.329621 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 142/284] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:37 2022] GET /identity/v3/users/a75fef9313fe4fe9b5081ca64ef1ff73 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:37.330399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4d5d5-9e17-402f-8fc8-13996f26e857 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.410692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f94b5-3b5b-4ae7-92d4-c79606e4f08e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:37.420252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:37.427891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4d5d5-9e17-402f-8fc8-13996f26e857 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:37.441952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:37.444694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4336ca8-3c78-4eaa-9b17-3c77b75c4c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.445306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4336ca8-3c78-4eaa-9b17-3c77b75c4c1a None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.445582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4336ca8-3c78-4eaa-9b17-3c77b75c4c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.445887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4336ca8-3c78-4eaa-9b17-3c77b75c4c1a None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.446541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4336ca8-3c78-4eaa-9b17-3c77b75c4c1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:37.448049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4336ca8-3c78-4eaa-9b17-3c77b75c4c1a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:37.464813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2885-49eb-4c28-b945-61827c6d33bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.465590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2885-49eb-4c28-b945-61827c6d33bb None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.465899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2885-49eb-4c28-b945-61827c6d33bb None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.466169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2885-49eb-4c28-b945-61827c6d33bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.476742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f2885-49eb-4c28-b945-61827c6d33bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-117f2885-49eb-4c28-b945-61827c6d33bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:37.483168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:37.484184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:37.485111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 143/285] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:37.488144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2885-49eb-4c28-b945-61827c6d33bb None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.529164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 143/286] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:29:37 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:29:37.530780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4336ca8-3c78-4eaa-9b17-3c77b75c4c1a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.606575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2885-49eb-4c28-b945-61827c6d33bb None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:37.656568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1884e8-5ab5-4d34-b05a-3c994fc3b8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.657341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1884e8-5ab5-4d34-b05a-3c994fc3b8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.657622 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1884e8-5ab5-4d34-b05a-3c994fc3b8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.657913 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1884e8-5ab5-4d34-b05a-3c994fc3b8e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.658657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1884e8-5ab5-4d34-b05a-3c994fc3b8e8 None admin] RBAC: Adding query filter params (role.id=d801ebea910445a7a6100b35e07630dd, user.id=2a8d693f7cf549e5bbe6558fd90f3fae, scope.system=all) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:37.658967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1884e8-5ab5-4d34-b05a-3c994fc3b8e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:37.660614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1884e8-5ab5-4d34-b05a-3c994fc3b8e8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:37.673863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 144/287] 10.222.0.42 () {56 vars in 1308 bytes} [Sat Sep 17 03:29:37 2022] GET /identity/v3/role_assignments?user.id=2a8d693f7cf549e5bbe6558fd90f3fae&scope.system=all&role.id=d801ebea910445a7a6100b35e07630dd => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:37.673863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1884e8-5ab5-4d34-b05a-3c994fc3b8e8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.685776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4336ca8-3c78-4eaa-9b17-3c77b75c4c1a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:37.736971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41d753-9f75-47b8-b317-839d12da9c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.737629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41d753-9f75-47b8-b317-839d12da9c97 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.737941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41d753-9f75-47b8-b317-839d12da9c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.738218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41d753-9f75-47b8-b317-839d12da9c97 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.739435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41d753-9f75-47b8-b317-839d12da9c97 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:37.740914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41d753-9f75-47b8-b317-839d12da9c97 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:37.744045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 144/288] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:37 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:37.745009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:37.755183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41d753-9f75-47b8-b317-839d12da9c97 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.816910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:37.839676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1884e8-5ab5-4d34-b05a-3c994fc3b8e8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:37.898342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826484c-6de9-4b67-8f60-ab577a6bde98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.899545 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826484c-6de9-4b67-8f60-ab577a6bde98 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.899961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826484c-6de9-4b67-8f60-ab577a6bde98 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.900396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826484c-6de9-4b67-8f60-ab577a6bde98 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.901224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826484c-6de9-4b67-8f60-ab577a6bde98 None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:37.901899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826484c-6de9-4b67-8f60-ab577a6bde98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:37.903632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826484c-6de9-4b67-8f60-ab577a6bde98 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:37.918460 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 145/289] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:37.926298 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41d753-9f75-47b8-b317-839d12da9c97 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:37.926982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826484c-6de9-4b67-8f60-ab577a6bde98 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:37.977220 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd69ca3-6220-45b4-bfec-3ab75008e548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:37.978309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd69ca3-6220-45b4-bfec-3ab75008e548 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:37.978797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd69ca3-6220-45b4-bfec-3ab75008e548 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:37.979309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd69ca3-6220-45b4-bfec-3ab75008e548 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb/users/a75fef9313fe4fe9b5081ca64ef1ff73/roles/ed414442e2d24f2f99e36d88ade1835e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:37.983144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd69ca3-6220-45b4-bfec-3ab75008e548 None admin] RBAC: Authorizing `identity:create_grant(project_id=e02b0fbedb42442bab7a31b8db209acb, user_id=a75fef9313fe4fe9b5081ca64ef1ff73, role_id=ed414442e2d24f2f99e36d88ade1835e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:37.984640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd69ca3-6220-45b4-bfec-3ab75008e548 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:38.059254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 145/290] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:29:37 2022] PUT /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb/users/a75fef9313fe4fe9b5081ca64ef1ff73/roles/ed414442e2d24f2f99e36d88ade1835e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:38.061679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:38.065507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826484c-6de9-4b67-8f60-ab577a6bde98 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:38.115331 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ca83b-b347-4ec8-9add-cd7c2932d281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:38.116420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca83b-b347-4ec8-9add-cd7c2932d281 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:38.116901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca83b-b347-4ec8-9add-cd7c2932d281 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:38.117371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca83b-b347-4ec8-9add-cd7c2932d281 None admin] PATH_INFO: `/v3/users/9fb32af3d8d5443eab6b1404d07a9743` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:38.119668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ca83b-b347-4ec8-9add-cd7c2932d281 None admin] RBAC: Authorizing `identity:get_user(user_id=9fb32af3d8d5443eab6b1404d07a9743)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:38.122459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ca83b-b347-4ec8-9add-cd7c2932d281 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:38.124486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 146/291] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:37 2022] GET /identity/v3/users/9fb32af3d8d5443eab6b1404d07a9743 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:38.130342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd69ca3-6220-45b4-bfec-3ab75008e548 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:38.293877 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:38.304306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd69ca3-6220-45b4-bfec-3ab75008e548 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:38.359290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b61fa-5811-4e20-ad7f-585e0269ac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:38.359910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b61fa-5811-4e20-ad7f-585e0269ac80 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:38.360187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b61fa-5811-4e20-ad7f-585e0269ac80 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:38.360456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b61fa-5811-4e20-ad7f-585e0269ac80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:38.476045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b61fa-5811-4e20-ad7f-585e0269ac80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:38.477548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b61fa-5811-4e20-ad7f-585e0269ac80 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:38.665458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:38.675722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 146/292] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:38.681355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ca83b-b347-4ec8-9add-cd7c2932d281 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:38.813855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ca83b-b347-4ec8-9add-cd7c2932d281 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:38.868846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee234a-657c-4c26-aa60-8db77d23a970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:38.869626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee234a-657c-4c26-aa60-8db77d23a970 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:38.869819 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee234a-657c-4c26-aa60-8db77d23a970 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:38.870131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee234a-657c-4c26-aa60-8db77d23a970 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:38.871474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee234a-657c-4c26-aa60-8db77d23a970 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:38.873295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee234a-657c-4c26-aa60-8db77d23a970 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:38.874983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 147/293] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:38 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:38.880773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b61fa-5811-4e20-ad7f-585e0269ac80 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:39.013424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b61fa-5811-4e20-ad7f-585e0269ac80 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:39.054026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352baaf1-098a-4788-bf42-13e7fd3a177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:39.054634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352baaf1-098a-4788-bf42-13e7fd3a177a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:39.054914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352baaf1-098a-4788-bf42-13e7fd3a177a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:39.055186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352baaf1-098a-4788-bf42-13e7fd3a177a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:39.055914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352baaf1-098a-4788-bf42-13e7fd3a177a None admin] RBAC: Adding query filter params (role.id=d801ebea910445a7a6100b35e07630dd, scope.project.id=e02b0fbedb42442bab7a31b8db209acb, user.id=9fb32af3d8d5443eab6b1404d07a9743) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:39.056221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352baaf1-098a-4788-bf42-13e7fd3a177a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:39.058304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352baaf1-098a-4788-bf42-13e7fd3a177a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:39.064762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 147/294] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:38 2022] GET /identity/v3/role_assignments?user.id=9fb32af3d8d5443eab6b1404d07a9743&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=d801ebea910445a7a6100b35e07630dd => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:40.278774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478620-b767-4f7e-bdca-9de54dda47ff None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:40.279294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478620-b767-4f7e-bdca-9de54dda47ff None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:40.279294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478620-b767-4f7e-bdca-9de54dda47ff None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:40.280663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 148/295] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:40.286077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3ed6-7c94-40af-b8be-72992efd9016 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:40.286502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3ed6-7c94-40af-b8be-72992efd9016 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:40.286674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3ed6-7c94-40af-b8be-72992efd9016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:40.305390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b6ae3ed6-7c94-40af-b8be-72992efd9016 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:40.306612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b6ae3ed6-7c94-40af-b8be-72992efd9016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:40.337624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1976f3f-b626-4a42-9e5a-67e716be9093 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:40.338271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1976f3f-b626-4a42-9e5a-67e716be9093 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:40.338634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1976f3f-b626-4a42-9e5a-67e716be9093 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:40.340786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 149/296] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:29:40.345917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78144ff9-ec71-449c-a490-b918a0d5c7be None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:40.346277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78144ff9-ec71-449c-a490-b918a0d5c7be None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:40.346631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78144ff9-ec71-449c-a490-b918a0d5c7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:40.365741 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:40.367310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-78144ff9-ec71-449c-a490-b918a0d5c7be None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:40.368585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-78144ff9-ec71-449c-a490-b918a0d5c7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:40.386454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 148/297] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:40.403550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:40.453837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 150/298] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:40.566555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b8891-2ec4-424a-bf0d-c96ae4f6f4fe None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:40.566847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b8891-2ec4-424a-bf0d-c96ae4f6f4fe None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:40.567030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b8891-2ec4-424a-bf0d-c96ae4f6f4fe None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:40.568753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 149/299] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:40.575486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c57b0f-2385-48f2-abff-2a731a7e700b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:40.575805 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c57b0f-2385-48f2-abff-2a731a7e700b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:40.576036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c57b0f-2385-48f2-abff-2a731a7e700b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:40.595091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-61c57b0f-2385-48f2-abff-2a731a7e700b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:40.596285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-61c57b0f-2385-48f2-abff-2a731a7e700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:40.663327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b8891-2ec4-424a-bf0d-c96ae4f6f4fe None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:40.664111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 151/300] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:40.685431 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69a571-98a5-44c2-aa91-b0d6115379d3 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:40.686090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69a571-98a5-44c2-aa91-b0d6115379d3 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:40.686569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69a571-98a5-44c2-aa91-b0d6115379d3 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:40.688389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 152/301] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:29:40.693402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97725e25-a6d0-4d09-be4c-bf5110b4c910 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:40.693948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97725e25-a6d0-4d09-be4c-bf5110b4c910 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:40.694412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97725e25-a6d0-4d09-be4c-bf5110b4c910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:40.715576 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-97725e25-a6d0-4d09-be4c-bf5110b4c910 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:40.717153 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-97725e25-a6d0-4d09-be4c-bf5110b4c910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:40.774967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b8891-2ec4-424a-bf0d-c96ae4f6f4fe None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:40.797691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 153/302] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:40.797691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97725e25-a6d0-4d09-be4c-bf5110b4c910 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:40.822480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc250222-cfdc-438a-8c4d-6b9178de1ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:40.823114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc250222-cfdc-438a-8c4d-6b9178de1ebb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:40.824000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc250222-cfdc-438a-8c4d-6b9178de1ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:40.824342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc250222-cfdc-438a-8c4d-6b9178de1ebb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:40.830290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:40.836292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc250222-cfdc-438a-8c4d-6b9178de1ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-bc250222-cfdc-438a-8c4d-6b9178de1ebb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:40.841983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:40.844266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:40.845207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:40.850574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 150/303] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:40.851739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc250222-cfdc-438a-8c4d-6b9178de1ebb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:40.903889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:40.926211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97725e25-a6d0-4d09-be4c-bf5110b4c910 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:40.963569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc250222-cfdc-438a-8c4d-6b9178de1ebb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:40.968325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be770290-ede8-4404-bed6-ea4527b1285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:40.968955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be770290-ede8-4404-bed6-ea4527b1285a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:40.969232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be770290-ede8-4404-bed6-ea4527b1285a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:40.969500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be770290-ede8-4404-bed6-ea4527b1285a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:40.980242 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be770290-ede8-4404-bed6-ea4527b1285a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-be770290-ede8-4404-bed6-ea4527b1285a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:40.985067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:40.986197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 154/304] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:40.987030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be770290-ede8-4404-bed6-ea4527b1285a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:41.009101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fe35b-29d9-405c-9a15-cee6bbbc307d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.011129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fe35b-29d9-405c-9a15-cee6bbbc307d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.011444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fe35b-29d9-405c-9a15-cee6bbbc307d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.011694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fe35b-29d9-405c-9a15-cee6bbbc307d None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.012358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fe35b-29d9-405c-9a15-cee6bbbc307d None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:41.012684 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fe35b-29d9-405c-9a15-cee6bbbc307d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:41.015948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fe35b-29d9-405c-9a15-cee6bbbc307d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:41.027150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 151/305] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:41.028073 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fe35b-29d9-405c-9a15-cee6bbbc307d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:41.091927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:41.107939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be770290-ede8-4404-bed6-ea4527b1285a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:41.142383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fe35b-29d9-405c-9a15-cee6bbbc307d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:41.148882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313f98e-bc56-4129-9947-7e0b46a61078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.149505 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313f98e-bc56-4129-9947-7e0b46a61078 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.149807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313f98e-bc56-4129-9947-7e0b46a61078 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.150086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313f98e-bc56-4129-9947-7e0b46a61078 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.151147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:41.151793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313f98e-bc56-4129-9947-7e0b46a61078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:41.153515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313f98e-bc56-4129-9947-7e0b46a61078 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:41.155051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 155/306] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:41.155652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313f98e-bc56-4129-9947-7e0b46a61078 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:41.176754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a17a93-51a0-4c3e-9304-a80e10546586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.177359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a17a93-51a0-4c3e-9304-a80e10546586 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.177643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a17a93-51a0-4c3e-9304-a80e10546586 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.177946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a17a93-51a0-4c3e-9304-a80e10546586 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.178609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a17a93-51a0-4c3e-9304-a80e10546586 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:41.178980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a17a93-51a0-4c3e-9304-a80e10546586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:41.180712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a17a93-51a0-4c3e-9304-a80e10546586 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:41.189298 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 152/307] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:41.190121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a17a93-51a0-4c3e-9304-a80e10546586 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:41.258509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313f98e-bc56-4129-9947-7e0b46a61078 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:41.289155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a17a93-51a0-4c3e-9304-a80e10546586 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:41.291230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed0fcb-ca72-4457-adb3-d86e7ef491a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.291845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed0fcb-ca72-4457-adb3-d86e7ef491a1 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.292675 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed0fcb-ca72-4457-adb3-d86e7ef491a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.292675 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed0fcb-ca72-4457-adb3-d86e7ef491a1 None admin] PATH_INFO: `/v3/users/2a8d693f7cf549e5bbe6558fd90f3fae` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.293621 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed0fcb-ca72-4457-adb3-d86e7ef491a1 None admin] RBAC: Authorizing `identity:get_user(user_id=2a8d693f7cf549e5bbe6558fd90f3fae)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:41.295087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed0fcb-ca72-4457-adb3-d86e7ef491a1 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:41.296890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 156/308] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:41 2022] GET /identity/v3/users/2a8d693f7cf549e5bbe6558fd90f3fae => generated 293 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:41.297554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed0fcb-ca72-4457-adb3-d86e7ef491a1 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:41.323029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:41.324873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab40279-0c8c-45cb-a7ae-1533881a69d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.325503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab40279-0c8c-45cb-a7ae-1533881a69d5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.325810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab40279-0c8c-45cb-a7ae-1533881a69d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.326084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab40279-0c8c-45cb-a7ae-1533881a69d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.336290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab40279-0c8c-45cb-a7ae-1533881a69d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4ab40279-0c8c-45cb-a7ae-1533881a69d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:41.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:41.342644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 153/309] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:41.358177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab40279-0c8c-45cb-a7ae-1533881a69d5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:41.378815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:41.397629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed0fcb-ca72-4457-adb3-d86e7ef491a1 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:41.434809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081eee7-b66a-4240-a6a4-38e13d6b4974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.435412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081eee7-b66a-4240-a6a4-38e13d6b4974 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.435693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081eee7-b66a-4240-a6a4-38e13d6b4974 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.435962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081eee7-b66a-4240-a6a4-38e13d6b4974 None admin] PATH_INFO: `/v3/groups` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.436586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081eee7-b66a-4240-a6a4-38e13d6b4974 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:41.438067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081eee7-b66a-4240-a6a4-38e13d6b4974 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:41.449448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab40279-0c8c-45cb-a7ae-1533881a69d5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:41.505100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 157/310] 10.222.0.42 () {60 vars in 1134 bytes} [Sat Sep 17 03:29:41 2022] POST /identity/v3/groups => generated 229 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:29:41.505912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081eee7-b66a-4240-a6a4-38e13d6b4974 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:41.540288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a40b4-1add-46a3-8756-84fee8ad5def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.540944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a40b4-1add-46a3-8756-84fee8ad5def None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.541227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a40b4-1add-46a3-8756-84fee8ad5def None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.541504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a40b4-1add-46a3-8756-84fee8ad5def None admin] PATH_INFO: `/v3/users/767e107ac92b47079d0446035702ee9f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.557033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a40b4-1add-46a3-8756-84fee8ad5def None admin] RBAC: Authorizing `identity:get_user(user_id=767e107ac92b47079d0446035702ee9f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:41.559614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a40b4-1add-46a3-8756-84fee8ad5def None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:41.566677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 154/311] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:41 2022] GET /identity/v3/users/767e107ac92b47079d0446035702ee9f => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:41.566677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a40b4-1add-46a3-8756-84fee8ad5def None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:41.611750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:41.631695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081eee7-b66a-4240-a6a4-38e13d6b4974 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:41.666054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:41.674688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ef21d-47e2-4668-8012-eeba80776c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.675807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ef21d-47e2-4668-8012-eeba80776c94 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.677125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ef21d-47e2-4668-8012-eeba80776c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.677401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ef21d-47e2-4668-8012-eeba80776c94 None admin] PATH_INFO: `/v3/system/users/2a8d693f7cf549e5bbe6558fd90f3fae/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.678888 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ef21d-47e2-4668-8012-eeba80776c94 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2a8d693f7cf549e5bbe6558fd90f3fae, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:41.679753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a40b4-1add-46a3-8756-84fee8ad5def None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:41.680284 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ef21d-47e2-4668-8012-eeba80776c94 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:41.716329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec79fc7-14a3-49fd-aa34-290051f756e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.717177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec79fc7-14a3-49fd-aa34-290051f756e5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.717461 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec79fc7-14a3-49fd-aa34-290051f756e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.717761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec79fc7-14a3-49fd-aa34-290051f756e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.718411 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec79fc7-14a3-49fd-aa34-290051f756e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:41.718709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec79fc7-14a3-49fd-aa34-290051f756e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:41.720103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec79fc7-14a3-49fd-aa34-290051f756e5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:41.728601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 155/312] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:41.729456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec79fc7-14a3-49fd-aa34-290051f756e5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:41.789838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 158/313] 10.222.0.42 () {58 vars in 1334 bytes} [Sat Sep 17 03:29:41 2022] PUT /identity/v3/system/users/2a8d693f7cf549e5bbe6558fd90f3fae/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:41.791058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ef21d-47e2-4668-8012-eeba80776c94 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:41.854174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec79fc7-14a3-49fd-aa34-290051f756e5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:41.906015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088908f0-9595-48f4-a179-f9a126c4611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.907336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088908f0-9595-48f4-a179-f9a126c4611a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.907824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088908f0-9595-48f4-a179-f9a126c4611a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.908584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088908f0-9595-48f4-a179-f9a126c4611a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.910129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088908f0-9595-48f4-a179-f9a126c4611a None admin] RBAC: Adding query filter params (role.id=ed414442e2d24f2f99e36d88ade1835e, user.id=767e107ac92b47079d0446035702ee9f, scope.system=all) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:41.910938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088908f0-9595-48f4-a179-f9a126c4611a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:41.912954 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088908f0-9595-48f4-a179-f9a126c4611a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:41.921062 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 156/314] 10.222.0.42 () {56 vars in 1308 bytes} [Sat Sep 17 03:29:41 2022] GET /identity/v3/role_assignments?user.id=767e107ac92b47079d0446035702ee9f&scope.system=all&role.id=ed414442e2d24f2f99e36d88ade1835e => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:41.931033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ef21d-47e2-4668-8012-eeba80776c94 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:41.982159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede0a20-55df-4e3a-9cb0-866117cc8508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:41.982807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede0a20-55df-4e3a-9cb0-866117cc8508 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:41.983084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede0a20-55df-4e3a-9cb0-866117cc8508 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:41.983352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede0a20-55df-4e3a-9cb0-866117cc8508 None admin] PATH_INFO: `/v3/users/a75fef9313fe4fe9b5081ca64ef1ff73` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:41.984513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede0a20-55df-4e3a-9cb0-866117cc8508 None admin] RBAC: Authorizing `identity:get_user(user_id=a75fef9313fe4fe9b5081ca64ef1ff73)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:41.986021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede0a20-55df-4e3a-9cb0-866117cc8508 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:41.988037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 159/315] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:41 2022] GET /identity/v3/users/a75fef9313fe4fe9b5081ca64ef1ff73 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:41.993677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088908f0-9595-48f4-a179-f9a126c4611a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:42.017072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:42.115483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088908f0-9595-48f4-a179-f9a126c4611a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:42.152816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b86cd-a38f-40cd-beed-0ebb91f0401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:42.153422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b86cd-a38f-40cd-beed-0ebb91f0401a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:42.153699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b86cd-a38f-40cd-beed-0ebb91f0401a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:42.153997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b86cd-a38f-40cd-beed-0ebb91f0401a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:42.225524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b86cd-a38f-40cd-beed-0ebb91f0401a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:42.226941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b86cd-a38f-40cd-beed-0ebb91f0401a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:42.243992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:42.332918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 157/316] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:42.338362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede0a20-55df-4e3a-9cb0-866117cc8508 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:42.357923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:42.456503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede0a20-55df-4e3a-9cb0-866117cc8508 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:42.519547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09bc24-6302-44b1-8bb1-8b41d6e0519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:42.520555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09bc24-6302-44b1-8bb1-8b41d6e0519a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:42.521163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09bc24-6302-44b1-8bb1-8b41d6e0519a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:42.521624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09bc24-6302-44b1-8bb1-8b41d6e0519a None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:42.523005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09bc24-6302-44b1-8bb1-8b41d6e0519a None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:42.524624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09bc24-6302-44b1-8bb1-8b41d6e0519a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:42.527358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 160/317] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:42 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:42.535122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b86cd-a38f-40cd-beed-0ebb91f0401a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:42.640402 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b86cd-a38f-40cd-beed-0ebb91f0401a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:42.655309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:42.675805 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9e74f-c3f7-4d21-8a58-bd7d9474b3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:42.676411 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9e74f-c3f7-4d21-8a58-bd7d9474b3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:42.679026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9e74f-c3f7-4d21-8a58-bd7d9474b3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:42.679026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9e74f-c3f7-4d21-8a58-bd7d9474b3d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:42.679026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9e74f-c3f7-4d21-8a58-bd7d9474b3d4 None admin] RBAC: Adding query filter params (role.id=ed414442e2d24f2f99e36d88ade1835e, scope.project.id=e02b0fbedb42442bab7a31b8db209acb, user.id=a75fef9313fe4fe9b5081ca64ef1ff73) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:42.679026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9e74f-c3f7-4d21-8a58-bd7d9474b3d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:42.679403 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9e74f-c3f7-4d21-8a58-bd7d9474b3d4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:42.686530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 158/318] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:29:42 2022] GET /identity/v3/role_assignments?user.id=a75fef9313fe4fe9b5081ca64ef1ff73&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=ed414442e2d24f2f99e36d88ade1835e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:43.761552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0edc0-2299-4416-afa0-3434d78d6d6a None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:43.762104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0edc0-2299-4416-afa0-3434d78d6d6a None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:43.762187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0edc0-2299-4416-afa0-3434d78d6d6a None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:43.763756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 161/319] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:43.769118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91200e8-9b44-4d54-833f-521dc0227d02 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:43.769543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91200e8-9b44-4d54-833f-521dc0227d02 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:43.769716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91200e8-9b44-4d54-833f-521dc0227d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:43.788281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e91200e8-9b44-4d54-833f-521dc0227d02 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:43.789466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e91200e8-9b44-4d54-833f-521dc0227d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:43.865147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 159/320] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:44.096641 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe43700-91bc-4ff1-9978-a1f252121f6e None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:44.096808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe43700-91bc-4ff1-9978-a1f252121f6e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:44.097085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe43700-91bc-4ff1-9978-a1f252121f6e None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:44.098645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 162/321] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:44.103961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f78f0f-e560-4e72-9729-874f177a6d78 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:44.104248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f78f0f-e560-4e72-9729-874f177a6d78 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:44.104725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f78f0f-e560-4e72-9729-874f177a6d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:44.123716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-00f78f0f-e560-4e72-9729-874f177a6d78 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:44.124880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-00f78f0f-e560-4e72-9729-874f177a6d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:44.144802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe43700-91bc-4ff1-9978-a1f252121f6e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:44.195551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 160/322] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:44.196889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7893dc1-fdd5-46b5-a482-1f6ff8241b5b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:44.197176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7893dc1-fdd5-46b5-a482-1f6ff8241b5b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:44.197449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7893dc1-fdd5-46b5-a482-1f6ff8241b5b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:44.199664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 161/323] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:44.204265 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351951b6-b0db-49ca-9985-5c33a7ddac47 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:44.204552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351951b6-b0db-49ca-9985-5c33a7ddac47 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:44.204851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351951b6-b0db-49ca-9985-5c33a7ddac47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:44.223229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-351951b6-b0db-49ca-9985-5c33a7ddac47 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:44.224427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-351951b6-b0db-49ca-9985-5c33a7ddac47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:44.229295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:44.250476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe43700-91bc-4ff1-9978-a1f252121f6e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:44.255751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:44.286350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18c3c0-7ad8-484d-973e-5b817d0b710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:44.287070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18c3c0-7ad8-484d-973e-5b817d0b710f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:44.287360 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18c3c0-7ad8-484d-973e-5b817d0b710f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:44.287634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18c3c0-7ad8-484d-973e-5b817d0b710f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:44.288741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18c3c0-7ad8-484d-973e-5b817d0b710f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:44.290214 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18c3c0-7ad8-484d-973e-5b817d0b710f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:44.292168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 163/324] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:29:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:44.297502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18c3c0-7ad8-484d-973e-5b817d0b710f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:44.297922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 162/325] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:44.399853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18c3c0-7ad8-484d-973e-5b817d0b710f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:44.434328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c328e7-113c-4210-b070-b31512a1e4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:44.434929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c328e7-113c-4210-b070-b31512a1e4ab None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:44.435206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c328e7-113c-4210-b070-b31512a1e4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:44.435476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c328e7-113c-4210-b070-b31512a1e4ab None admin] PATH_INFO: `/v3/groups` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:44.436102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c328e7-113c-4210-b070-b31512a1e4ab None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:44.437526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c328e7-113c-4210-b070-b31512a1e4ab None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:44.465594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351951b6-b0db-49ca-9985-5c33a7ddac47 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:44.549752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 164/326] 10.222.0.42 () {60 vars in 1134 bytes} [Sat Sep 17 03:29:44 2022] POST /identity/v3/groups => generated 226 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:29:44.568958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c328e7-113c-4210-b070-b31512a1e4ab None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:44.580635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351951b6-b0db-49ca-9985-5c33a7ddac47 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:44.583030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:44.621686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c849ea-6eb6-4199-95ef-9e2f0988a112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:44.622667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c849ea-6eb6-4199-95ef-9e2f0988a112 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:44.623156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c849ea-6eb6-4199-95ef-9e2f0988a112 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:44.623582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c849ea-6eb6-4199-95ef-9e2f0988a112 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:44.636387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c849ea-6eb6-4199-95ef-9e2f0988a112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-89c849ea-6eb6-4199-95ef-9e2f0988a112 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:44.641903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:44.643041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:44.644039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:44.646290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 163/327] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:44.654254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c849ea-6eb6-4199-95ef-9e2f0988a112 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:44.688056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c328e7-113c-4210-b070-b31512a1e4ab None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:44.688619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:44.729767 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32d95a-dc53-407f-aae1-3623ab31ebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:44.730918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32d95a-dc53-407f-aae1-3623ab31ebca None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:44.730918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32d95a-dc53-407f-aae1-3623ab31ebca None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:44.730918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32d95a-dc53-407f-aae1-3623ab31ebca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:44.749205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32d95a-dc53-407f-aae1-3623ab31ebca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-2c32d95a-dc53-407f-aae1-3623ab31ebca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:44.755444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:44.756349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:44.757241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 165/328] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:44.763458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32d95a-dc53-407f-aae1-3623ab31ebca None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:44.779376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c849ea-6eb6-4199-95ef-9e2f0988a112 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:44.830453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed11ac-1e89-4647-a378-e575df91d0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:44.831991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed11ac-1e89-4647-a378-e575df91d0dc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:44.831991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed11ac-1e89-4647-a378-e575df91d0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:44.831991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed11ac-1e89-4647-a378-e575df91d0dc None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:44.832431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed11ac-1e89-4647-a378-e575df91d0dc None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:44.832795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed11ac-1e89-4647-a378-e575df91d0dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:44.834302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed11ac-1e89-4647-a378-e575df91d0dc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:44.844429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 164/329] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:44.846233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:44.851058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed11ac-1e89-4647-a378-e575df91d0dc None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:44.880516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32d95a-dc53-407f-aae1-3623ab31ebca None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:44.926481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389970a8-0e4f-435c-a7bc-225cf0f0b941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:44.927288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389970a8-0e4f-435c-a7bc-225cf0f0b941 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:44.928306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389970a8-0e4f-435c-a7bc-225cf0f0b941 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:44.928306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389970a8-0e4f-435c-a7bc-225cf0f0b941 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:44.928486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389970a8-0e4f-435c-a7bc-225cf0f0b941 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:44.928786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389970a8-0e4f-435c-a7bc-225cf0f0b941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:44.930350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389970a8-0e4f-435c-a7bc-225cf0f0b941 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:44.941978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 166/330] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:44.947181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389970a8-0e4f-435c-a7bc-225cf0f0b941 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:44.964332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:44.977400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed11ac-1e89-4647-a378-e575df91d0dc None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:45.020908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803c541-2943-4249-bb59-0fb78922bf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:45.021537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803c541-2943-4249-bb59-0fb78922bf7d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:45.021861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803c541-2943-4249-bb59-0fb78922bf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:45.022139 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803c541-2943-4249-bb59-0fb78922bf7d None admin] PATH_INFO: `/v3/users/2a8d693f7cf549e5bbe6558fd90f3fae` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:45.024015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803c541-2943-4249-bb59-0fb78922bf7d None admin] RBAC: Authorizing `identity:get_user(user_id=2a8d693f7cf549e5bbe6558fd90f3fae)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:45.025464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803c541-2943-4249-bb59-0fb78922bf7d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:45.033179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 165/331] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:44 2022] GET /identity/v3/users/2a8d693f7cf549e5bbe6558fd90f3fae => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:45.034459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803c541-2943-4249-bb59-0fb78922bf7d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:45.094078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389970a8-0e4f-435c-a7bc-225cf0f0b941 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:45.132686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:45.141173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9875a-5ada-4ce5-ba85-56b094ec21e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:45.141887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9875a-5ada-4ce5-ba85-56b094ec21e0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:45.142170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9875a-5ada-4ce5-ba85-56b094ec21e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:45.142445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9875a-5ada-4ce5-ba85-56b094ec21e0 None admin] PATH_INFO: `/v3/users/767e107ac92b47079d0446035702ee9f` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:45.146043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9875a-5ada-4ce5-ba85-56b094ec21e0 None admin] RBAC: Authorizing `identity:get_user(user_id=767e107ac92b47079d0446035702ee9f)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:45.146043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9875a-5ada-4ce5-ba85-56b094ec21e0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:45.147072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 167/332] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:44 2022] GET /identity/v3/users/767e107ac92b47079d0446035702ee9f => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:45.154697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9875a-5ada-4ce5-ba85-56b094ec21e0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:45.185293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803c541-2943-4249-bb59-0fb78922bf7d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:45.236507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b37d1-8ea0-4fbb-9caf-85e127996520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:45.237937 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b37d1-8ea0-4fbb-9caf-85e127996520 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:45.238426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b37d1-8ea0-4fbb-9caf-85e127996520 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:45.238895 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b37d1-8ea0-4fbb-9caf-85e127996520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:45.239857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b37d1-8ea0-4fbb-9caf-85e127996520 None admin] RBAC: Adding query filter params (role.id=d801ebea910445a7a6100b35e07630dd, user.id=2a8d693f7cf549e5bbe6558fd90f3fae, scope.system=all) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:45.240391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b37d1-8ea0-4fbb-9caf-85e127996520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:45.242358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b37d1-8ea0-4fbb-9caf-85e127996520 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:45.268093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 166/333] 10.222.0.42 () {56 vars in 1308 bytes} [Sat Sep 17 03:29:45 2022] GET /identity/v3/role_assignments?user.id=2a8d693f7cf549e5bbe6558fd90f3fae&scope.system=all&role.id=d801ebea910445a7a6100b35e07630dd => generated 510 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:45.299829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9875a-5ada-4ce5-ba85-56b094ec21e0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:45.340987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622c303-8355-4856-b2be-03f3550ce9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:45.341935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622c303-8355-4856-b2be-03f3550ce9f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:45.342351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622c303-8355-4856-b2be-03f3550ce9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:45.342761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622c303-8355-4856-b2be-03f3550ce9f9 None admin] PATH_INFO: `/v3/system/users/767e107ac92b47079d0446035702ee9f/roles/ed414442e2d24f2f99e36d88ade1835e` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:45.344519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622c303-8355-4856-b2be-03f3550ce9f9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=767e107ac92b47079d0446035702ee9f, role_id=ed414442e2d24f2f99e36d88ade1835e)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:45.346157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622c303-8355-4856-b2be-03f3550ce9f9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:45.433555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 168/334] 10.222.0.42 () {58 vars in 1334 bytes} [Sat Sep 17 03:29:45 2022] PUT /identity/v3/system/users/767e107ac92b47079d0446035702ee9f/roles/ed414442e2d24f2f99e36d88ade1835e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:47.572699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb41ae-643d-4efb-929f-e6d10359c7dc None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:47.574751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb41ae-643d-4efb-929f-e6d10359c7dc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:47.574751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb41ae-643d-4efb-929f-e6d10359c7dc None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:47.575500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 167/335] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:47.580978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc942bf-92b0-4c27-bad2-7a276321ad77 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:47.581496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc942bf-92b0-4c27-bad2-7a276321ad77 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:47.581496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc942bf-92b0-4c27-bad2-7a276321ad77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:47.601597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ee0e-011d-4c80-84ba-2e92c2bc53df None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:47.601971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ee0e-011d-4c80-84ba-2e92c2bc53df None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:47.602250 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ee0e-011d-4c80-84ba-2e92c2bc53df None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:47.603835 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 168/336] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:47.606796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-bfc942bf-92b0-4c27-bad2-7a276321ad77 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:47.608005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bfc942bf-92b0-4c27-bad2-7a276321ad77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:47.614244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828e09-423f-405f-a8c6-2da5fa709046 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:47.614528 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828e09-423f-405f-a8c6-2da5fa709046 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:47.614800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828e09-423f-405f-a8c6-2da5fa709046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:47.646828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c4828e09-423f-405f-a8c6-2da5fa709046 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:47.648251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c4828e09-423f-405f-a8c6-2da5fa709046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:47.671204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:47.714524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:47.726104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 169/337] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:47.768907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 169/338] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:47.902620 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901302b-5d01-4d19-9689-ee3ee51e5f38 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:47.902930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901302b-5d01-4d19-9689-ee3ee51e5f38 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:47.903175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901302b-5d01-4d19-9689-ee3ee51e5f38 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:47.904805 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 170/339] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:47.910394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21292e75-1090-4ee1-b773-14096dcc2370 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:47.910689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21292e75-1090-4ee1-b773-14096dcc2370 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:47.910970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21292e75-1090-4ee1-b773-14096dcc2370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:47.930997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-21292e75-1090-4ee1-b773-14096dcc2370 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:47.932201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-21292e75-1090-4ee1-b773-14096dcc2370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:48.005084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 170/340] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:48.266794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901302b-5d01-4d19-9689-ee3ee51e5f38 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:48.277158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21292e75-1090-4ee1-b773-14096dcc2370 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:48.404050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:48.430650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21292e75-1090-4ee1-b773-14096dcc2370 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:48.460577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901302b-5d01-4d19-9689-ee3ee51e5f38 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:48.473607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85dd9-cfc4-4d76-a692-7526bfc3fde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:48.476569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85dd9-cfc4-4d76-a692-7526bfc3fde1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:48.476569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85dd9-cfc4-4d76-a692-7526bfc3fde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:48.476569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85dd9-cfc4-4d76-a692-7526bfc3fde1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:48.488120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d85dd9-cfc4-4d76-a692-7526bfc3fde1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c0d85dd9-cfc4-4d76-a692-7526bfc3fde1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:48.493355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:48.494276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:48.495169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:48.497915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 171/341] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:48.499043 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85dd9-cfc4-4d76-a692-7526bfc3fde1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:48.535957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d9b36-2615-48bf-aa51-75ded60140bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:48.536516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d9b36-2615-48bf-aa51-75ded60140bc None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:48.536793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d9b36-2615-48bf-aa51-75ded60140bc None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:48.537061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d9b36-2615-48bf-aa51-75ded60140bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:48.558907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d9b36-2615-48bf-aa51-75ded60140bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-cb8d9b36-2615-48bf-aa51-75ded60140bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:48.564005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:48.564898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:48.565736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 171/342] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:48.566424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d9b36-2615-48bf-aa51-75ded60140bc None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:48.614410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85dd9-cfc4-4d76-a692-7526bfc3fde1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:48.654357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13765d63-b3b2-4c89-82f2-df5d769ebdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:48.655524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13765d63-b3b2-4c89-82f2-df5d769ebdb8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:48.656022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13765d63-b3b2-4c89-82f2-df5d769ebdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:48.656457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13765d63-b3b2-4c89-82f2-df5d769ebdb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:48.657715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:48.670602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13765d63-b3b2-4c89-82f2-df5d769ebdb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-13765d63-b3b2-4c89-82f2-df5d769ebdb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:48.676221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:48.677213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:48.678169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:48.681036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 172/343] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:48.686824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:48.687335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13765d63-b3b2-4c89-82f2-df5d769ebdb8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:48.715061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d9b36-2615-48bf-aa51-75ded60140bc None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:48.753317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af1d11-52bb-4d93-84e1-f650e89cb9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:48.753686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af1d11-52bb-4d93-84e1-f650e89cb9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:48.754067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af1d11-52bb-4d93-84e1-f650e89cb9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:48.754344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af1d11-52bb-4d93-84e1-f650e89cb9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:48.755017 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af1d11-52bb-4d93-84e1-f650e89cb9e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:48.755381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af1d11-52bb-4d93-84e1-f650e89cb9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:48.756710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af1d11-52bb-4d93-84e1-f650e89cb9e5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:48.765489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 172/344] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:48.766396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af1d11-52bb-4d93-84e1-f650e89cb9e5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:48.806535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13765d63-b3b2-4c89-82f2-df5d769ebdb8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:48.857751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63141d-54aa-4f85-bd7c-f64fd1301de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:48.857751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63141d-54aa-4f85-bd7c-f64fd1301de1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:48.857751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63141d-54aa-4f85-bd7c-f64fd1301de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:48.857751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63141d-54aa-4f85-bd7c-f64fd1301de1 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:48.858206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63141d-54aa-4f85-bd7c-f64fd1301de1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:48.858206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63141d-54aa-4f85-bd7c-f64fd1301de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:48.858532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63141d-54aa-4f85-bd7c-f64fd1301de1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:48.868229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 173/345] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:48.868884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63141d-54aa-4f85-bd7c-f64fd1301de1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:48.883899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af1d11-52bb-4d93-84e1-f650e89cb9e5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:48.903429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:48.924391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91489bd-1996-4eb9-9d6e-c2c36468a3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:48.925004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91489bd-1996-4eb9-9d6e-c2c36468a3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:48.925420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91489bd-1996-4eb9-9d6e-c2c36468a3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:48.925700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91489bd-1996-4eb9-9d6e-c2c36468a3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:48.926577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91489bd-1996-4eb9-9d6e-c2c36468a3f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:48.926929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91489bd-1996-4eb9-9d6e-c2c36468a3f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:48.928310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91489bd-1996-4eb9-9d6e-c2c36468a3f7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:48.939778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 173/346] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:48.940302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:48.945964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7536832-cacd-4e17-ad9b-d660b3096f8b None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:48.946250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7536832-cacd-4e17-ad9b-d660b3096f8b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:48.953267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7536832-cacd-4e17-ad9b-d660b3096f8b None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:48.953267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 174/347] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:48.953267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7536832-cacd-4e17-ad9b-d660b3096f8b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:48.992609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63141d-54aa-4f85-bd7c-f64fd1301de1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:49.027654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883ad8b-a796-4101-9b0f-54da173dcae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:49.028669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883ad8b-a796-4101-9b0f-54da173dcae0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:49.028669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883ad8b-a796-4101-9b0f-54da173dcae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:49.028949 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883ad8b-a796-4101-9b0f-54da173dcae0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:49.092134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7536832-cacd-4e17-ad9b-d660b3096f8b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:49.118638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883ad8b-a796-4101-9b0f-54da173dcae0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:49.122177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883ad8b-a796-4101-9b0f-54da173dcae0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:49.142669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c82f79-c176-486d-9097-666faed08a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:49.143289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c82f79-c176-486d-9097-666faed08a51 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:49.143565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c82f79-c176-486d-9097-666faed08a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:49.143836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c82f79-c176-486d-9097-666faed08a51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:49.206354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:49.221061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:49.239379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c82f79-c176-486d-9097-666faed08a51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:49.240917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c82f79-c176-486d-9097-666faed08a51 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:49.299310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 174/348] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:49.301424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109643ed-1b62-4d86-a199-4067d8e5c4d7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:49.301934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109643ed-1b62-4d86-a199-4067d8e5c4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:49.302407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109643ed-1b62-4d86-a199-4067d8e5c4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:49.325417 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-109643ed-1b62-4d86-a199-4067d8e5c4d7 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:49.327867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-109643ed-1b62-4d86-a199-4067d8e5c4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:49.382545 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 175/349] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:49.383372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c82f79-c176-486d-9097-666faed08a51 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:49.407775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 175/350] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:49.408961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109643ed-1b62-4d86-a199-4067d8e5c4d7 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:49.480604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:49.503321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c82f79-c176-486d-9097-666faed08a51 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:49.527651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:49.545480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109643ed-1b62-4d86-a199-4067d8e5c4d7 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:49.547205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48a082-4791-44ba-86c3-f186e2aaba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:49.547816 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48a082-4791-44ba-86c3-f186e2aaba7b None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:49.548091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48a082-4791-44ba-86c3-f186e2aaba7b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:49.548359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48a082-4791-44ba-86c3-f186e2aaba7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:49.598445 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b974de7-873d-4975-8530-cb3c0a596741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:49.599609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b974de7-873d-4975-8530-cb3c0a596741 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:49.600083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b974de7-873d-4975-8530-cb3c0a596741 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:49.600564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b974de7-873d-4975-8530-cb3c0a596741 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:49.601936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b974de7-873d-4975-8530-cb3c0a596741 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:49.603718 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b974de7-873d-4975-8530-cb3c0a596741 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:49.607049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 176/351] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:49 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:49.608576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79db0-ce65-4306-9113-349f69fe44f5 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:49.609057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79db0-ce65-4306-9113-349f69fe44f5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:49.609516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79db0-ce65-4306-9113-349f69fe44f5 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:49.611322 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 177/352] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:49.612214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af79db0-ce65-4306-9113-349f69fe44f5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:49.661018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48a082-4791-44ba-86c3-f186e2aaba7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:49.661018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48a082-4791-44ba-86c3-f186e2aaba7b None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:49.748653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af79db0-ce65-4306-9113-349f69fe44f5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:49.756080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:49.785576 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 176/353] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:49.786469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48a082-4791-44ba-86c3-f186e2aaba7b None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:49.807298 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a982361-34ae-4b2a-9c09-7fe7017c3d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:49.807977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a982361-34ae-4b2a-9c09-7fe7017c3d3d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:49.808398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a982361-34ae-4b2a-9c09-7fe7017c3d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:49.808677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a982361-34ae-4b2a-9c09-7fe7017c3d3d None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:49.810197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a982361-34ae-4b2a-9c09-7fe7017c3d3d None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:49.811319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a982361-34ae-4b2a-9c09-7fe7017c3d3d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:49.812981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 178/354] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:49 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:49.815005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456c481-27b3-49d0-ba0a-871fb9217eee None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:49.815288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456c481-27b3-49d0-ba0a-871fb9217eee None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:49.815562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456c481-27b3-49d0-ba0a-871fb9217eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:49.835082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8456c481-27b3-49d0-ba0a-871fb9217eee None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:49.836257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8456c481-27b3-49d0-ba0a-871fb9217eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:49.858169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:49.910950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48a082-4791-44ba-86c3-f186e2aaba7b None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:49.915018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 179/355] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:49.915807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456c481-27b3-49d0-ba0a-871fb9217eee None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:49.957540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6b951-34a9-425d-8511-79d0c0d4ba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:49.959136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b951-34a9-425d-8511-79d0c0d4ba82 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:49.959245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b951-34a9-425d-8511-79d0c0d4ba82 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:49.959524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b951-34a9-425d-8511-79d0c0d4ba82 None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:49.968512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6b951-34a9-425d-8511-79d0c0d4ba82 None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:49.970128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6b951-34a9-425d-8511-79d0c0d4ba82 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:49.972249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 177/356] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:49 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:49.973463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f47e4-1b3f-4c90-9e87-a72bc67d8c58 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:49.973758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f47e4-1b3f-4c90-9e87-a72bc67d8c58 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:49.974040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f47e4-1b3f-4c90-9e87-a72bc67d8c58 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:49.975370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 178/357] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:49.975927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f47e4-1b3f-4c90-9e87-a72bc67d8c58 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:50.032238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:50.051486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456c481-27b3-49d0-ba0a-871fb9217eee None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:50.094346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3e85b-d624-4906-b997-e4b62b929fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:50.095013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3e85b-d624-4906-b997-e4b62b929fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:50.095815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3e85b-d624-4906-b997-e4b62b929fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:50.096111 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3e85b-d624-4906-b997-e4b62b929fb8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:50.110839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3e85b-d624-4906-b997-e4b62b929fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:50.112686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f47e4-1b3f-4c90-9e87-a72bc67d8c58 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-6df3e85b-d624-4906-b997-e4b62b929fb8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:50.115702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:50.116685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:50.117487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 180/358] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:50.118220 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3e85b-d624-4906-b997-e4b62b929fb8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:50.134959 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:50.152901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e7d97-7e5e-4f52-86f9-d82a5b76f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:50.153529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e7d97-7e5e-4f52-86f9-d82a5b76f064 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:50.153842 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e7d97-7e5e-4f52-86f9-d82a5b76f064 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:50.154112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e7d97-7e5e-4f52-86f9-d82a5b76f064 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:50.155297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e7d97-7e5e-4f52-86f9-d82a5b76f064 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:50.156733 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e7d97-7e5e-4f52-86f9-d82a5b76f064 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:50.159225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 179/359] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:49 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:50.160544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f22db-251c-4987-b969-7f95c5fa5a32 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:50.160544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f22db-251c-4987-b969-7f95c5fa5a32 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:50.160544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f22db-251c-4987-b969-7f95c5fa5a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:50.187983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-7f7f22db-251c-4987-b969-7f95c5fa5a32 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:50.189100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7f7f22db-251c-4987-b969-7f95c5fa5a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:50.246419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3e85b-d624-4906-b997-e4b62b929fb8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:50.270679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 180/360] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:50.271811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f22db-251c-4987-b969-7f95c5fa5a32 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:50.286762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24073fb1-25d1-4a2b-9e09-ff738af20044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:50.287508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24073fb1-25d1-4a2b-9e09-ff738af20044 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:50.287789 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24073fb1-25d1-4a2b-9e09-ff738af20044 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:50.288068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24073fb1-25d1-4a2b-9e09-ff738af20044 None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:50.289187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24073fb1-25d1-4a2b-9e09-ff738af20044 None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:50.290937 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24073fb1-25d1-4a2b-9e09-ff738af20044 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:50.293401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 181/361] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:50 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:50.294158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24073fb1-25d1-4a2b-9e09-ff738af20044 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:50.319013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:50.391709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f22db-251c-4987-b969-7f95c5fa5a32 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:50.400742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:50.422432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24073fb1-25d1-4a2b-9e09-ff738af20044 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:50.434213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07fbb6-ae1d-4733-a0ba-17b3d9860dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:50.435346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07fbb6-ae1d-4733-a0ba-17b3d9860dbe None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:50.435594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07fbb6-ae1d-4733-a0ba-17b3d9860dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:50.435930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07fbb6-ae1d-4733-a0ba-17b3d9860dbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:50.437025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07fbb6-ae1d-4733-a0ba-17b3d9860dbe None admin] RBAC: Adding query filter params (group.id=b5477168f623485b9b9c0ad3c9f02b32, role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:50.437395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07fbb6-ae1d-4733-a0ba-17b3d9860dbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:50.439695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07fbb6-ae1d-4733-a0ba-17b3d9860dbe None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:50.446349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 181/362] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Sep 17 03:29:50 2022] GET /identity/v3/role_assignments?group.id=b5477168f623485b9b9c0ad3c9f02b32&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:50.447075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07fbb6-ae1d-4733-a0ba-17b3d9860dbe None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:50.465752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481407c-cf1b-478b-bc93-58357f728ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:50.466597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481407c-cf1b-478b-bc93-58357f728ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:50.466987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481407c-cf1b-478b-bc93-58357f728ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:50.467358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481407c-cf1b-478b-bc93-58357f728ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:50.468166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481407c-cf1b-478b-bc93-58357f728ad3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:50.468564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481407c-cf1b-478b-bc93-58357f728ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:50.470112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481407c-cf1b-478b-bc93-58357f728ad3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:50.480069 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 182/363] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:50.481049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481407c-cf1b-478b-bc93-58357f728ad3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:50.569574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07fbb6-ae1d-4733-a0ba-17b3d9860dbe None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:50.572850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:50.607945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481407c-cf1b-478b-bc93-58357f728ad3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:50.622969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821aeeb-3252-47de-8658-d6940985186b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:50.623708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821aeeb-3252-47de-8658-d6940985186b None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:50.623993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821aeeb-3252-47de-8658-d6940985186b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:50.624266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821aeeb-3252-47de-8658-d6940985186b None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:50.625416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821aeeb-3252-47de-8658-d6940985186b None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:50.627537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821aeeb-3252-47de-8658-d6940985186b None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:50.628882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 182/364] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:50 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:50.629691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821aeeb-3252-47de-8658-d6940985186b None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:50.655095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855f8ad-d017-4560-8582-0e5690461cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:50.655738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855f8ad-d017-4560-8582-0e5690461cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:50.656035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855f8ad-d017-4560-8582-0e5690461cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:50.656308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855f8ad-d017-4560-8582-0e5690461cd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:50.674101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855f8ad-d017-4560-8582-0e5690461cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f855f8ad-d017-4560-8582-0e5690461cd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:50.679606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:50.680693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:50.681692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:50.685704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 183/365] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:50.686654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855f8ad-d017-4560-8582-0e5690461cd9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:50.691837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:50.763347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821aeeb-3252-47de-8658-d6940985186b None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:50.816991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226839f8-029e-4854-a562-863e9a4e5a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:50.817597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226839f8-029e-4854-a562-863e9a4e5a68 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:50.817962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226839f8-029e-4854-a562-863e9a4e5a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:50.818240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226839f8-029e-4854-a562-863e9a4e5a68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:50.818976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226839f8-029e-4854-a562-863e9a4e5a68 None admin] RBAC: Adding query filter params (group.id=b5477168f623485b9b9c0ad3c9f02b32, role.id=d801ebea910445a7a6100b35e07630dd, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:50.819490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226839f8-029e-4854-a562-863e9a4e5a68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:50.820721 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226839f8-029e-4854-a562-863e9a4e5a68 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:50.830401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855f8ad-d017-4560-8582-0e5690461cd9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:50.834850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 183/366] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Sep 17 03:29:50 2022] GET /identity/v3/role_assignments?group.id=b5477168f623485b9b9c0ad3c9f02b32&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=d801ebea910445a7a6100b35e07630dd => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:50.834850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226839f8-029e-4854-a562-863e9a4e5a68 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:50.876826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:50.890031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d2e64-a6b7-4083-850a-1af161621b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:50.892135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d2e64-a6b7-4083-850a-1af161621b1f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:50.892135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d2e64-a6b7-4083-850a-1af161621b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:50.892135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d2e64-a6b7-4083-850a-1af161621b1f None admin] PATH_INFO: `/v3/users/767e107ac92b47079d0446035702ee9f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:50.893149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d2e64-a6b7-4083-850a-1af161621b1f None admin] RBAC: Authorizing `identity:get_user(user_id=767e107ac92b47079d0446035702ee9f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:50.895159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d2e64-a6b7-4083-850a-1af161621b1f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:50.896921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 184/367] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:29:50 2022] GET /identity/v3/users/767e107ac92b47079d0446035702ee9f => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:50.898326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d2e64-a6b7-4083-850a-1af161621b1f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:50.969749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226839f8-029e-4854-a562-863e9a4e5a68 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:50.997036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:51.016530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080d837-4de1-4c63-ba51-9e10bb28c18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:51.017105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080d837-4de1-4c63-ba51-9e10bb28c18a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:51.017384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080d837-4de1-4c63-ba51-9e10bb28c18a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:51.017788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080d837-4de1-4c63-ba51-9e10bb28c18a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:51.037749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080d837-4de1-4c63-ba51-9e10bb28c18a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:51.042903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d2e64-a6b7-4083-850a-1af161621b1f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-0080d837-4de1-4c63-ba51-9e10bb28c18a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:51.044186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:51.045059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:51.045916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 184/368] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:29:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:51.046593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080d837-4de1-4c63-ba51-9e10bb28c18a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:51.091211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edbf09-9b09-49dd-9c0c-88ffc2369567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:51.091579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edbf09-9b09-49dd-9c0c-88ffc2369567 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:51.091847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edbf09-9b09-49dd-9c0c-88ffc2369567 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:51.092120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edbf09-9b09-49dd-9c0c-88ffc2369567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:51.092849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edbf09-9b09-49dd-9c0c-88ffc2369567 None admin] RBAC: Adding query filter params (group.id=b5477168f623485b9b9c0ad3c9f02b32, role.id=d801ebea910445a7a6100b35e07630dd, scope.project.id=e02b0fbedb42442bab7a31b8db209acb) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:51.093155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edbf09-9b09-49dd-9c0c-88ffc2369567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:51.095579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edbf09-9b09-49dd-9c0c-88ffc2369567 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:51.103630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 185/369] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Sep 17 03:29:50 2022] GET /identity/v3/role_assignments?group.id=b5477168f623485b9b9c0ad3c9f02b32&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=d801ebea910445a7a6100b35e07630dd => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:51.105009 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edbf09-9b09-49dd-9c0c-88ffc2369567 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:51.235939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:51.244942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080d837-4de1-4c63-ba51-9e10bb28c18a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:51.250658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edbf09-9b09-49dd-9c0c-88ffc2369567 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:51.285829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbcb73-ecfe-41c3-b5d4-3f9bb4180ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:51.286594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbcb73-ecfe-41c3-b5d4-3f9bb4180ffd None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:51.286943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbcb73-ecfe-41c3-b5d4-3f9bb4180ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:51.287292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbcb73-ecfe-41c3-b5d4-3f9bb4180ffd None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:51.287987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbcb73-ecfe-41c3-b5d4-3f9bb4180ffd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:51.288294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbcb73-ecfe-41c3-b5d4-3f9bb4180ffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:51.289701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbcb73-ecfe-41c3-b5d4-3f9bb4180ffd None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:51.294158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbafe9-1ece-4273-b600-d4e9999a844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:51.294784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbafe9-1ece-4273-b600-d4e9999a844e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:51.295061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbafe9-1ece-4273-b600-d4e9999a844e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:51.295328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbafe9-1ece-4273-b600-d4e9999a844e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:51.296275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbafe9-1ece-4273-b600-d4e9999a844e None admin] RBAC: Adding query filter params (role.id=ed414442e2d24f2f99e36d88ade1835e, user.id=767e107ac92b47079d0446035702ee9f, scope.system=all) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:51.296588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbafe9-1ece-4273-b600-d4e9999a844e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:51.298077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbafe9-1ece-4273-b600-d4e9999a844e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:51.299462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 185/370] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:51.300457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbcb73-ecfe-41c3-b5d4-3f9bb4180ffd None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:51.301754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:51.314598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 186/371] 10.222.0.42 () {56 vars in 1308 bytes} [Sat Sep 17 03:29:51 2022] GET /identity/v3/role_assignments?user.id=767e107ac92b47079d0446035702ee9f&scope.system=all&role.id=ed414442e2d24f2f99e36d88ade1835e => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:51.315820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbafe9-1ece-4273-b600-d4e9999a844e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:51.504466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbcb73-ecfe-41c3-b5d4-3f9bb4180ffd None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:51.545980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbafe9-1ece-4273-b600-d4e9999a844e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:51.552346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b065e27-778c-4316-afdf-1cecea0a1f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:51.553159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b065e27-778c-4316-afdf-1cecea0a1f86 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:51.553534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b065e27-778c-4316-afdf-1cecea0a1f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:51.553941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b065e27-778c-4316-afdf-1cecea0a1f86 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:51.554749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b065e27-778c-4316-afdf-1cecea0a1f86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:51.555162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b065e27-778c-4316-afdf-1cecea0a1f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:51.556643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b065e27-778c-4316-afdf-1cecea0a1f86 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:51.572546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 186/372] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:51.579665 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b065e27-778c-4316-afdf-1cecea0a1f86 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:51.625648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0573503-a405-4a80-8f7e-6734a8e50c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:51.629579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0573503-a405-4a80-8f7e-6734a8e50c05 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:51.630030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0573503-a405-4a80-8f7e-6734a8e50c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:51.630475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0573503-a405-4a80-8f7e-6734a8e50c05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:51.633648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:51.764247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:51.777518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0573503-a405-4a80-8f7e-6734a8e50c05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:51.779003 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0573503-a405-4a80-8f7e-6734a8e50c05 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:51.779869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b065e27-778c-4316-afdf-1cecea0a1f86 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:51.833764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae6197-00db-4280-954f-daa2b366b737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:51.834628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae6197-00db-4280-954f-daa2b366b737 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:51.835052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae6197-00db-4280-954f-daa2b366b737 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:51.835488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae6197-00db-4280-954f-daa2b366b737 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:51.937156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae6197-00db-4280-954f-daa2b366b737 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:51.938964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae6197-00db-4280-954f-daa2b366b737 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:51.967327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:51.998158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 187/373] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:52.004397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0573503-a405-4a80-8f7e-6734a8e50c05 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:52.087202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 187/374] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:29:52.092469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae6197-00db-4280-954f-daa2b366b737 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:52.150052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0573503-a405-4a80-8f7e-6734a8e50c05 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:52.152333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:52.221115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaf8e7-1410-4abc-a8d0-10cf41f7a705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:52.221773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaf8e7-1410-4abc-a8d0-10cf41f7a705 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:52.222166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaf8e7-1410-4abc-a8d0-10cf41f7a705 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:52.222437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaf8e7-1410-4abc-a8d0-10cf41f7a705 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:52.223929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaf8e7-1410-4abc-a8d0-10cf41f7a705 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:52.225609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaf8e7-1410-4abc-a8d0-10cf41f7a705 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:52.227252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 188/375] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:52 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:52.234478 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaf8e7-1410-4abc-a8d0-10cf41f7a705 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:52.255425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae6197-00db-4280-954f-daa2b366b737 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:52.294162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:52.323872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d121dd-4afb-4b20-bd17-03613e0d2179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:52.324510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d121dd-4afb-4b20-bd17-03613e0d2179 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:52.324789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d121dd-4afb-4b20-bd17-03613e0d2179 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:52.325061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d121dd-4afb-4b20-bd17-03613e0d2179 None admin] PATH_INFO: `/v3/projects/545b5824a9314a3a81e4babed059fa06` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:52.330060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d121dd-4afb-4b20-bd17-03613e0d2179 None admin] RBAC: Authorizing `identity:get_project(project_id=545b5824a9314a3a81e4babed059fa06)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:52.331518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d121dd-4afb-4b20-bd17-03613e0d2179 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:52.337614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 188/376] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:52 2022] GET /identity/v3/projects/545b5824a9314a3a81e4babed059fa06 => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:52.359923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d121dd-4afb-4b20-bd17-03613e0d2179 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:52.431010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaf8e7-1410-4abc-a8d0-10cf41f7a705 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:52.471564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004160d9-47f8-49f5-8a8c-001cf9f8a714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:52.472196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004160d9-47f8-49f5-8a8c-001cf9f8a714 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:52.472479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004160d9-47f8-49f5-8a8c-001cf9f8a714 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:52.472759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004160d9-47f8-49f5-8a8c-001cf9f8a714 None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:52.474022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004160d9-47f8-49f5-8a8c-001cf9f8a714 None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:52.475481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004160d9-47f8-49f5-8a8c-001cf9f8a714 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:52.480361 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 189/377] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:52 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:52.488702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004160d9-47f8-49f5-8a8c-001cf9f8a714 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:52.536347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d121dd-4afb-4b20-bd17-03613e0d2179 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:52.550397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:52.604072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bec26-e963-4143-bfde-3737e3ba91da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:52.605154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bec26-e963-4143-bfde-3737e3ba91da None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:52.605420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bec26-e963-4143-bfde-3737e3ba91da None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:52.605857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bec26-e963-4143-bfde-3737e3ba91da None admin] PATH_INFO: `/v3/groups/d291e8482c46487a940dc344619d7980` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:52.617220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bec26-e963-4143-bfde-3737e3ba91da None admin] RBAC: Authorizing `identity:get_group(group_id=d291e8482c46487a940dc344619d7980)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:52.618693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bec26-e963-4143-bfde-3737e3ba91da None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:52.622418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 189/378] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:52 2022] GET /identity/v3/groups/d291e8482c46487a940dc344619d7980 => generated 229 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:52.627585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bec26-e963-4143-bfde-3737e3ba91da None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:52.676429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004160d9-47f8-49f5-8a8c-001cf9f8a714 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:52.722401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:52.743229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868067f4-764d-42fa-8924-a271a8d7b789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:52.743836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868067f4-764d-42fa-8924-a271a8d7b789 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:52.744110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868067f4-764d-42fa-8924-a271a8d7b789 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:52.744380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868067f4-764d-42fa-8924-a271a8d7b789 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:52.748689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868067f4-764d-42fa-8924-a271a8d7b789 None admin] RBAC: Adding query filter params (group.id=b5477168f623485b9b9c0ad3c9f02b32, role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba, scope.project.id=e02b0fbedb42442bab7a31b8db209acb) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:52.748689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868067f4-764d-42fa-8924-a271a8d7b789 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:52.749598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868067f4-764d-42fa-8924-a271a8d7b789 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:52.762704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 190/379] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Sep 17 03:29:52 2022] GET /identity/v3/role_assignments?group.id=b5477168f623485b9b9c0ad3c9f02b32&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 260 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:52.793442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bec26-e963-4143-bfde-3737e3ba91da None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:52.850504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bff16-e39e-4ff6-9e20-e26d3f09ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:52.851598 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bff16-e39e-4ff6-9e20-e26d3f09ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:52.852122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bff16-e39e-4ff6-9e20-e26d3f09ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:52.852601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bff16-e39e-4ff6-9e20-e26d3f09ccf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:52.853602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bff16-e39e-4ff6-9e20-e26d3f09ccf4 None admin] RBAC: Adding query filter params (group.id=d291e8482c46487a940dc344619d7980, role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=545b5824a9314a3a81e4babed059fa06) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:52.855042 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bff16-e39e-4ff6-9e20-e26d3f09ccf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:52.856782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bff16-e39e-4ff6-9e20-e26d3f09ccf4 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:52.863727 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 190/380] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Sep 17 03:29:52 2022] GET /identity/v3/role_assignments?group.id=d291e8482c46487a940dc344619d7980&scope.project.id=545b5824a9314a3a81e4babed059fa06&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:53.308727 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708f464-ff4a-43fe-b16a-958bc099a0fa None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:53.309278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708f464-ff4a-43fe-b16a-958bc099a0fa None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:53.309278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708f464-ff4a-43fe-b16a-958bc099a0fa None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:53.311042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 191/381] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:53.316094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389a2a-7e55-4e3f-8dab-70337b7ec707 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:53.316367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389a2a-7e55-4e3f-8dab-70337b7ec707 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:53.316645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389a2a-7e55-4e3f-8dab-70337b7ec707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:53.342403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-3a389a2a-7e55-4e3f-8dab-70337b7ec707 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:53.343623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-3a389a2a-7e55-4e3f-8dab-70337b7ec707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:53.420079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 191/382] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:53.734524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91634d-5017-4d83-8f3a-0ec8238c5c3b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:53.734894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91634d-5017-4d83-8f3a-0ec8238c5c3b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:53.735195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91634d-5017-4d83-8f3a-0ec8238c5c3b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:53.736800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 192/383] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:53.741970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9f671-ece9-414f-8a40-67c39c7e0aef None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:53.742259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9f671-ece9-414f-8a40-67c39c7e0aef None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:53.742534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9f671-ece9-414f-8a40-67c39c7e0aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:53.761539 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a6a9f671-ece9-414f-8a40-67c39c7e0aef None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:53.766831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a6a9f671-ece9-414f-8a40-67c39c7e0aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:53.829288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:53.848226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91634d-5017-4d83-8f3a-0ec8238c5c3b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:53.861564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 192/384] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:53.918077 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952d2c6-92a7-4e13-97f1-7743e456365a None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:53.918384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952d2c6-92a7-4e13-97f1-7743e456365a None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:53.918628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952d2c6-92a7-4e13-97f1-7743e456365a None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:53.920153 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 193/385] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:29:53.925402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b3b6d-daa0-455c-a67d-d4429346a29f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:53.925676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b3b6d-daa0-455c-a67d-d4429346a29f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:53.925988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b3b6d-daa0-455c-a67d-d4429346a29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:53.946349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d33b3b6d-daa0-455c-a67d-d4429346a29f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:53.948081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d33b3b6d-daa0-455c-a67d-d4429346a29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:54.038038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 194/386] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:54.039859 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:54.067333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91634d-5017-4d83-8f3a-0ec8238c5c3b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:54.125864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d155a58-3818-4662-92e8-67acd2c158dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:54.126496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d155a58-3818-4662-92e8-67acd2c158dd None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:54.126773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d155a58-3818-4662-92e8-67acd2c158dd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:54.127045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d155a58-3818-4662-92e8-67acd2c158dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:54.146999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d155a58-3818-4662-92e8-67acd2c158dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-6d155a58-3818-4662-92e8-67acd2c158dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:54.152191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:54.153188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:54.154512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 193/387] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:54.159412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b3b6d-daa0-455c-a67d-d4429346a29f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:54.193197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d155a58-3818-4662-92e8-67acd2c158dd None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:54.301774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b3b6d-daa0-455c-a67d-d4429346a29f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:54.310054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:54.340121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f760a-3948-4916-9774-5c3b3bdc1380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:54.340878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f760a-3948-4916-9774-5c3b3bdc1380 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:54.341155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f760a-3948-4916-9774-5c3b3bdc1380 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:54.341427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f760a-3948-4916-9774-5c3b3bdc1380 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:54.342116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f760a-3948-4916-9774-5c3b3bdc1380 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:54.342416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f760a-3948-4916-9774-5c3b3bdc1380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:54.343777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f760a-3948-4916-9774-5c3b3bdc1380 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:54.353019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 195/388] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:54.358873 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f760a-3948-4916-9774-5c3b3bdc1380 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:54.411088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d155a58-3818-4662-92e8-67acd2c158dd None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:54.448910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141de07-a9d7-4cbf-8821-ca203cb2b5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:54.449931 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141de07-a9d7-4cbf-8821-ca203cb2b5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:54.450430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141de07-a9d7-4cbf-8821-ca203cb2b5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:54.450980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141de07-a9d7-4cbf-8821-ca203cb2b5b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:54.462269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141de07-a9d7-4cbf-8821-ca203cb2b5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4141de07-a9d7-4cbf-8821-ca203cb2b5b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:54.469739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:54.470690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:54.471488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:54.474323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 194/389] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:54.475423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141de07-a9d7-4cbf-8821-ca203cb2b5b0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:54.483251 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f760a-3948-4916-9774-5c3b3bdc1380 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:54.495812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:54.529802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079beed-bf59-438f-8d29-ddd64af68176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:54.530784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079beed-bf59-438f-8d29-ddd64af68176 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:54.531719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079beed-bf59-438f-8d29-ddd64af68176 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:54.532189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079beed-bf59-438f-8d29-ddd64af68176 None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:54.534234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079beed-bf59-438f-8d29-ddd64af68176 None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:54.536147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079beed-bf59-438f-8d29-ddd64af68176 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:54.538998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 196/390] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:54 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:54.540116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079beed-bf59-438f-8d29-ddd64af68176 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:54.587411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:54.612074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141de07-a9d7-4cbf-8821-ca203cb2b5b0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:54.648749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7192e-ed42-4a1c-a7b5-ab346263c4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:54.649720 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7192e-ed42-4a1c-a7b5-ab346263c4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:54.650242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7192e-ed42-4a1c-a7b5-ab346263c4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:54.650722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7192e-ed42-4a1c-a7b5-ab346263c4f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:54.670272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7192e-ed42-4a1c-a7b5-ab346263c4f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:54.672665 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079beed-bf59-438f-8d29-ddd64af68176 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-85b7192e-ed42-4a1c-a7b5-ab346263c4f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:54.675887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:54.676860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:54.677652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:54.680214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 195/391] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:54.681310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7192e-ed42-4a1c-a7b5-ab346263c4f9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:54.717058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66936b-a61a-40cd-a691-0f4b61a45782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:54.717793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66936b-a61a-40cd-a691-0f4b61a45782 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:54.718082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66936b-a61a-40cd-a691-0f4b61a45782 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:54.718409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66936b-a61a-40cd-a691-0f4b61a45782 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:54.719185 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66936b-a61a-40cd-a691-0f4b61a45782 None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:54.719473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66936b-a61a-40cd-a691-0f4b61a45782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:54.720947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66936b-a61a-40cd-a691-0f4b61a45782 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:54.735301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 197/392] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:54.735301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66936b-a61a-40cd-a691-0f4b61a45782 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:54.766693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:54.791957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7192e-ed42-4a1c-a7b5-ab346263c4f9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:54.828938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcd2c2-c913-41f2-acaa-9e81385a6115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:54.829677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcd2c2-c913-41f2-acaa-9e81385a6115 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:54.830135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcd2c2-c913-41f2-acaa-9e81385a6115 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:54.830503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcd2c2-c913-41f2-acaa-9e81385a6115 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:54.831723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcd2c2-c913-41f2-acaa-9e81385a6115 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:54.833264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcd2c2-c913-41f2-acaa-9e81385a6115 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:54.835314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 196/393] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:54 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:54.836252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcd2c2-c913-41f2-acaa-9e81385a6115 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:54.851969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:54.857679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66936b-a61a-40cd-a691-0f4b61a45782 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:54.893906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a69c8-e942-4d99-8dd2-6f182a593bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:54.894919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a69c8-e942-4d99-8dd2-6f182a593bce None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:54.895344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a69c8-e942-4d99-8dd2-6f182a593bce None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:54.895771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a69c8-e942-4d99-8dd2-6f182a593bce None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:54.896579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a69c8-e942-4d99-8dd2-6f182a593bce None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:54.897037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a69c8-e942-4d99-8dd2-6f182a593bce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:54.898626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a69c8-e942-4d99-8dd2-6f182a593bce None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:54.912807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 198/394] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:54.913991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a69c8-e942-4d99-8dd2-6f182a593bce None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:54.942933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcd2c2-c913-41f2-acaa-9e81385a6115 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:54.978889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3e57f-cfeb-422e-933a-377e7ca443de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:54.980131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3e57f-cfeb-422e-933a-377e7ca443de None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:54.980486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3e57f-cfeb-422e-933a-377e7ca443de None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:54.980825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3e57f-cfeb-422e-933a-377e7ca443de None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:54.982254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3e57f-cfeb-422e-933a-377e7ca443de None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:54.983727 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3e57f-cfeb-422e-933a-377e7ca443de None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:54.985679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 197/395] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:54 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:54.990894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3e57f-cfeb-422e-933a-377e7ca443de None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:55.003512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:55.027269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a69c8-e942-4d99-8dd2-6f182a593bce None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:55.064822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9c8cf-f44b-46f0-9bee-75f3a2d10b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:55.065654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c8cf-f44b-46f0-9bee-75f3a2d10b68 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:55.065980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c8cf-f44b-46f0-9bee-75f3a2d10b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:55.066356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c8cf-f44b-46f0-9bee-75f3a2d10b68 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/groups/b5477168f623485b9b9c0ad3c9f02b32/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:55.069434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9c8cf-f44b-46f0-9bee-75f3a2d10b68 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6, group_id=b5477168f623485b9b9c0ad3c9f02b32, role_id=ec3ad2e5f7314c3b8d35826cfd81b5ba)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:55.071036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9c8cf-f44b-46f0-9bee-75f3a2d10b68 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:55.099966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3e57f-cfeb-422e-933a-377e7ca443de None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:55.137366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5b060-fccb-4190-8b9c-b81025f732a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:55.137988 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5b060-fccb-4190-8b9c-b81025f732a9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:55.138245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5b060-fccb-4190-8b9c-b81025f732a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:55.138518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5b060-fccb-4190-8b9c-b81025f732a9 None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:55.139826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5b060-fccb-4190-8b9c-b81025f732a9 None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:55.141261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5b060-fccb-4190-8b9c-b81025f732a9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:55.142467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 199/396] 10.222.0.42 () {58 vars in 1442 bytes} [Sat Sep 17 03:29:54 2022] PUT /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/groups/b5477168f623485b9b9c0ad3c9f02b32/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:55.147636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 198/397] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:54 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:55.148591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9c8cf-f44b-46f0-9bee-75f3a2d10b68 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:55.154923 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5b060-fccb-4190-8b9c-b81025f732a9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:55.181754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:55.294490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9c8cf-f44b-46f0-9bee-75f3a2d10b68 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:55.310786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:55.342027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026cb5c-553d-4b75-ae76-5e175c46293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:55.342416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5b060-fccb-4190-8b9c-b81025f732a9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:55.343307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026cb5c-553d-4b75-ae76-5e175c46293b None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:55.344208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026cb5c-553d-4b75-ae76-5e175c46293b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:55.345110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026cb5c-553d-4b75-ae76-5e175c46293b None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:55.347503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026cb5c-553d-4b75-ae76-5e175c46293b None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:55.349183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026cb5c-553d-4b75-ae76-5e175c46293b None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:55.351994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 200/398] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:55 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:55.357858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026cb5c-553d-4b75-ae76-5e175c46293b None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:55.381898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcbb7c-5482-4b2d-9a1f-e2ba079712e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:55.382522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcbb7c-5482-4b2d-9a1f-e2ba079712e5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:55.382812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcbb7c-5482-4b2d-9a1f-e2ba079712e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:55.383085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcbb7c-5482-4b2d-9a1f-e2ba079712e5 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:55.384197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcbb7c-5482-4b2d-9a1f-e2ba079712e5 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:55.386128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcbb7c-5482-4b2d-9a1f-e2ba079712e5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:55.391618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 199/399] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:55 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:55.396793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcbb7c-5482-4b2d-9a1f-e2ba079712e5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:55.488495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:55.502771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026cb5c-553d-4b75-ae76-5e175c46293b None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:55.537937 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcbb7c-5482-4b2d-9a1f-e2ba079712e5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:55.538503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdfd29-8e9d-4942-a9af-b55e111b8b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:55.538984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdfd29-8e9d-4942-a9af-b55e111b8b2e None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:55.539258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdfd29-8e9d-4942-a9af-b55e111b8b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:55.539533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdfd29-8e9d-4942-a9af-b55e111b8b2e None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/groups/b5477168f623485b9b9c0ad3c9f02b32/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:55.541574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdfd29-8e9d-4942-a9af-b55e111b8b2e None admin] RBAC: Authorizing `identity:create_grant(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6, group_id=b5477168f623485b9b9c0ad3c9f02b32, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:55.543167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdfd29-8e9d-4942-a9af-b55e111b8b2e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:55.577544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82201e-38e4-4f70-9b79-c8279a72d5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:55.578426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82201e-38e4-4f70-9b79-c8279a72d5bc None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:55.578751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82201e-38e4-4f70-9b79-c8279a72d5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:55.579127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82201e-38e4-4f70-9b79-c8279a72d5bc None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb/groups/b5477168f623485b9b9c0ad3c9f02b32/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:55.581163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82201e-38e4-4f70-9b79-c8279a72d5bc None admin] RBAC: Authorizing `identity:create_grant(project_id=e02b0fbedb42442bab7a31b8db209acb, group_id=b5477168f623485b9b9c0ad3c9f02b32, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:55.582732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82201e-38e4-4f70-9b79-c8279a72d5bc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:55.614323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 201/400] 10.222.0.42 () {58 vars in 1442 bytes} [Sat Sep 17 03:29:55 2022] PUT /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6/groups/b5477168f623485b9b9c0ad3c9f02b32/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:55.615663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8555ad-0c8e-41b5-bbc6-388f06849473 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:55.616050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8555ad-0c8e-41b5-bbc6-388f06849473 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:55.616433 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8555ad-0c8e-41b5-bbc6-388f06849473 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:55.620790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 202/401] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:55.629790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5be172-9508-4eb8-a885-4a43f4afe213 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:55.629790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5be172-9508-4eb8-a885-4a43f4afe213 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:55.629790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5be172-9508-4eb8-a885-4a43f4afe213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:55.649810 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-1e5be172-9508-4eb8-a885-4a43f4afe213 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:55.651194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1e5be172-9508-4eb8-a885-4a43f4afe213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:55.684734 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 200/402] 10.222.0.42 () {58 vars in 1442 bytes} [Sat Sep 17 03:29:55 2022] PUT /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb/groups/b5477168f623485b9b9c0ad3c9f02b32/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:55.686315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe172a2-d294-42d7-a602-3ec05699cc35 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:55.686750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe172a2-d294-42d7-a602-3ec05699cc35 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:55.687560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe172a2-d294-42d7-a602-3ec05699cc35 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:55.689013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 201/403] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:55.695509 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a2126-5831-4b92-9a26-a3c8fd43081d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:55.695926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a2126-5831-4b92-9a26-a3c8fd43081d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:55.696311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a2126-5831-4b92-9a26-a3c8fd43081d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:55.724291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:55.739227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-dc8a2126-5831-4b92-9a26-a3c8fd43081d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:55.740645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dc8a2126-5831-4b92-9a26-a3c8fd43081d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:55.755996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 203/404] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:55.878188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 202/405] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:29:56.067344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5be172-9508-4eb8-a885-4a43f4afe213 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:56.154932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:56.173778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a2126-5831-4b92-9a26-a3c8fd43081d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:56.178582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5be172-9508-4eb8-a885-4a43f4afe213 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:56.219149 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562603e-8601-4205-832f-8bb190906f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:56.220113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562603e-8601-4205-832f-8bb190906f7a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:56.220621 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562603e-8601-4205-832f-8bb190906f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:56.220868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562603e-8601-4205-832f-8bb190906f7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:56.232702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562603e-8601-4205-832f-8bb190906f7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-3562603e-8601-4205-832f-8bb190906f7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:56.237599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:56.238541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:56.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 204/406] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:29:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:56.245284 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562603e-8601-4205-832f-8bb190906f7a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:56.279760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a2126-5831-4b92-9a26-a3c8fd43081d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:56.315926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640682ee-a767-4c6d-8051-fb39ef82266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:56.316562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640682ee-a767-4c6d-8051-fb39ef82266a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:56.316845 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640682ee-a767-4c6d-8051-fb39ef82266a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:56.317127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640682ee-a767-4c6d-8051-fb39ef82266a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:56.329125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640682ee-a767-4c6d-8051-fb39ef82266a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-640682ee-a767-4c6d-8051-fb39ef82266a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:56.334115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:56.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:56.335897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:56.337754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:56.340481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 203/407] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:29:56.341879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640682ee-a767-4c6d-8051-fb39ef82266a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:56.356993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562603e-8601-4205-832f-8bb190906f7a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:56.393887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767acb69-c714-40fe-87c0-c2dd1c91c8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:56.394532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767acb69-c714-40fe-87c0-c2dd1c91c8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:56.394822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767acb69-c714-40fe-87c0-c2dd1c91c8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:56.395071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767acb69-c714-40fe-87c0-c2dd1c91c8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:56.395739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767acb69-c714-40fe-87c0-c2dd1c91c8a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:56.396037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767acb69-c714-40fe-87c0-c2dd1c91c8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:56.397404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767acb69-c714-40fe-87c0-c2dd1c91c8a8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:56.403244 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:56.407329 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 205/408] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:56.412818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767acb69-c714-40fe-87c0-c2dd1c91c8a8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:56.450049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640682ee-a767-4c6d-8051-fb39ef82266a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:56.486120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2f0b2-8dce-4dd2-a5bd-5fdca8d3fde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:56.486791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2f0b2-8dce-4dd2-a5bd-5fdca8d3fde7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:56.487083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2f0b2-8dce-4dd2-a5bd-5fdca8d3fde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:56.487366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2f0b2-8dce-4dd2-a5bd-5fdca8d3fde7 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:56.488041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2f0b2-8dce-4dd2-a5bd-5fdca8d3fde7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:56.488353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2f0b2-8dce-4dd2-a5bd-5fdca8d3fde7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:56.490081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2f0b2-8dce-4dd2-a5bd-5fdca8d3fde7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:56.503944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 204/409] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:56.505288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2f0b2-8dce-4dd2-a5bd-5fdca8d3fde7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:56.570201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:56.571564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767acb69-c714-40fe-87c0-c2dd1c91c8a8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:56.612423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2f0b2-8dce-4dd2-a5bd-5fdca8d3fde7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:56.639007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35580c39-83d6-4d72-8a7e-96415ba10676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:56.639644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35580c39-83d6-4d72-8a7e-96415ba10676 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:56.639988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35580c39-83d6-4d72-8a7e-96415ba10676 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:56.640268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35580c39-83d6-4d72-8a7e-96415ba10676 None admin] PATH_INFO: `/v3/groups/d291e8482c46487a940dc344619d7980` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:56.641406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35580c39-83d6-4d72-8a7e-96415ba10676 None admin] RBAC: Authorizing `identity:get_group(group_id=d291e8482c46487a940dc344619d7980)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:56.642904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35580c39-83d6-4d72-8a7e-96415ba10676 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:56.644786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 206/410] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:56 2022] GET /identity/v3/groups/d291e8482c46487a940dc344619d7980 => generated 229 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:56.650148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35580c39-83d6-4d72-8a7e-96415ba10676 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:56.656649 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0321353-c36c-4b7f-bf8f-03550a78c369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:56.657291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0321353-c36c-4b7f-bf8f-03550a78c369 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:56.657579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0321353-c36c-4b7f-bf8f-03550a78c369 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:56.657893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0321353-c36c-4b7f-bf8f-03550a78c369 None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:56.659068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0321353-c36c-4b7f-bf8f-03550a78c369 None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:56.660644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0321353-c36c-4b7f-bf8f-03550a78c369 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:56.667811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 205/411] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:56 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:56.669110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0321353-c36c-4b7f-bf8f-03550a78c369 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:56.728112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:56.766211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35580c39-83d6-4d72-8a7e-96415ba10676 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:56.801912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4ae3e-9a10-42ef-bf59-91ffbcb39601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:56.802231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4ae3e-9a10-42ef-bf59-91ffbcb39601 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:56.802458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4ae3e-9a10-42ef-bf59-91ffbcb39601 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:56.805891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4ae3e-9a10-42ef-bf59-91ffbcb39601 None admin] PATH_INFO: `/v3/projects/545b5824a9314a3a81e4babed059fa06` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:56.805891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4ae3e-9a10-42ef-bf59-91ffbcb39601 None admin] RBAC: Authorizing `identity:get_project(project_id=545b5824a9314a3a81e4babed059fa06)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:56.805891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4ae3e-9a10-42ef-bf59-91ffbcb39601 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:56.807963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 207/412] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:56 2022] GET /identity/v3/projects/545b5824a9314a3a81e4babed059fa06 => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:56.813823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4ae3e-9a10-42ef-bf59-91ffbcb39601 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:56.824071 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0321353-c36c-4b7f-bf8f-03550a78c369 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:56.904975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:56.924255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4ae3e-9a10-42ef-bf59-91ffbcb39601 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:56.939902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e05db9-b819-4370-8592-14938f79cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:56.940880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05db9-b819-4370-8592-14938f79cc10 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:56.941371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05db9-b819-4370-8592-14938f79cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:56.941890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05db9-b819-4370-8592-14938f79cc10 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:56.943276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e05db9-b819-4370-8592-14938f79cc10 None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:56.944913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e05db9-b819-4370-8592-14938f79cc10 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:56.946790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 206/413] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:56 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:56.952965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e05db9-b819-4370-8592-14938f79cc10 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:56.965060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ad138-67eb-45a4-a02d-731cc61d5149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:56.965736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ad138-67eb-45a4-a02d-731cc61d5149 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:56.966038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ad138-67eb-45a4-a02d-731cc61d5149 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:56.966311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ad138-67eb-45a4-a02d-731cc61d5149 None admin] PATH_INFO: `/v3/projects/545b5824a9314a3a81e4babed059fa06/groups/d291e8482c46487a940dc344619d7980/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:56.973748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ad138-67eb-45a4-a02d-731cc61d5149 None admin] RBAC: Authorizing `identity:create_grant(project_id=545b5824a9314a3a81e4babed059fa06, group_id=d291e8482c46487a940dc344619d7980, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:56.973748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ad138-67eb-45a4-a02d-731cc61d5149 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:56.973748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:57.045694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 208/414] 10.222.0.42 () {58 vars in 1442 bytes} [Sat Sep 17 03:29:56 2022] PUT /identity/v3/projects/545b5824a9314a3a81e4babed059fa06/groups/d291e8482c46487a940dc344619d7980/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:29:57.075191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e05db9-b819-4370-8592-14938f79cc10 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:57.119033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cab087-760c-48f0-9ffb-cf1c57f5a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:57.120369 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cab087-760c-48f0-9ffb-cf1c57f5a5a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:57.120883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cab087-760c-48f0-9ffb-cf1c57f5a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:57.121371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cab087-760c-48f0-9ffb-cf1c57f5a5a6 None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb/groups/b5477168f623485b9b9c0ad3c9f02b32/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:57.123835 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cab087-760c-48f0-9ffb-cf1c57f5a5a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e02b0fbedb42442bab7a31b8db209acb, group_id=b5477168f623485b9b9c0ad3c9f02b32, role_id=ec3ad2e5f7314c3b8d35826cfd81b5ba)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:57.125502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cab087-760c-48f0-9ffb-cf1c57f5a5a6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:57.210958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 207/415] 10.222.0.42 () {58 vars in 1442 bytes} [Sat Sep 17 03:29:56 2022] PUT /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb/groups/b5477168f623485b9b9c0ad3c9f02b32/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:29:57.855022 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebafa6-b17b-4172-bf91-da293581b61c None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:57.855564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebafa6-b17b-4172-bf91-da293581b61c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:57.855564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebafa6-b17b-4172-bf91-da293581b61c None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:57.857199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 209/416] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:57.863999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb346f17-5aec-4452-ada3-876834cfffa6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:57.864411 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb346f17-5aec-4452-ada3-876834cfffa6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:57.864599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb346f17-5aec-4452-ada3-876834cfffa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:57.887034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-bb346f17-5aec-4452-ada3-876834cfffa6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:57.888230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bb346f17-5aec-4452-ada3-876834cfffa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:57.957525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:57.977766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 208/417] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:58.363990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebafa6-b17b-4172-bf91-da293581b61c None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:58.504130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebafa6-b17b-4172-bf91-da293581b61c None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:58.521362 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4200ac6-ee2a-4b28-83d7-a28800f99b57 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:58.521649 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4200ac6-ee2a-4b28-83d7-a28800f99b57 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:58.521957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4200ac6-ee2a-4b28-83d7-a28800f99b57 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:58.523555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 209/418] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:58.539587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c5bb3-3ee2-448c-aae5-0794089ce03d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:58.539872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c5bb3-3ee2-448c-aae5-0794089ce03d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:58.540211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c5bb3-3ee2-448c-aae5-0794089ce03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:58.564367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6d0c5bb3-3ee2-448c-aae5-0794089ce03d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:58.565551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6d0c5bb3-3ee2-448c-aae5-0794089ce03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:58.575709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3a02f-75e2-467a-a6fb-90ad04c0638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:58.576347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3a02f-75e2-467a-a6fb-90ad04c0638e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:58.576626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3a02f-75e2-467a-a6fb-90ad04c0638e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:58.576899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3a02f-75e2-467a-a6fb-90ad04c0638e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:58.600073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3a02f-75e2-467a-a6fb-90ad04c0638e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-33f3a02f-75e2-467a-a6fb-90ad04c0638e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:58.605449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:58.606381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:29:58.607220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 210/419] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:29:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:58.614713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3a02f-75e2-467a-a6fb-90ad04c0638e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:58.689332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 210/420] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 17 03:29:58.723359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:58.816011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3a02f-75e2-467a-a6fb-90ad04c0638e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:58.856122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7680204-9878-4ddc-bacd-893edca10cd1 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:58.856441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7680204-9878-4ddc-bacd-893edca10cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:58.856690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7680204-9878-4ddc-bacd-893edca10cd1 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:58.862414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 211/421] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:58.865538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e628e9a-c5b8-4532-8956-148cdffd9fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:58.866722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e628e9a-c5b8-4532-8956-148cdffd9fac None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:58.867157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4667e7c-0220-4dd6-bbe0-cd38865a134e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:58.867526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4667e7c-0220-4dd6-bbe0-cd38865a134e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:58.867713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4667e7c-0220-4dd6-bbe0-cd38865a134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:58.867936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e628e9a-c5b8-4532-8956-148cdffd9fac None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:58.868407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e628e9a-c5b8-4532-8956-148cdffd9fac None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:58.869264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e628e9a-c5b8-4532-8956-148cdffd9fac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:58.870398 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e628e9a-c5b8-4532-8956-148cdffd9fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:58.871952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e628e9a-c5b8-4532-8956-148cdffd9fac None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:58.880562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 211/422] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:29:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:29:58.887283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e628e9a-c5b8-4532-8956-148cdffd9fac None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:58.897155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c4667e7c-0220-4dd6-bbe0-cd38865a134e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:29:58.898440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c4667e7c-0220-4dd6-bbe0-cd38865a134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:29:58.989936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 212/423] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:29:59.048844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e628e9a-c5b8-4532-8956-148cdffd9fac None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:59.068661 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:59.087313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1592a7e-21b9-4fde-b4d9-18a5fe48ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:59.087937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1592a7e-21b9-4fde-b4d9-18a5fe48ce0a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.088213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1592a7e-21b9-4fde-b4d9-18a5fe48ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.089148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1592a7e-21b9-4fde-b4d9-18a5fe48ce0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.140061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4667e7c-0220-4dd6-bbe0-cd38865a134e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:59.187026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1592a7e-21b9-4fde-b4d9-18a5fe48ce0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:59.188833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1592a7e-21b9-4fde-b4d9-18a5fe48ce0a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:59.227221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:59.329352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 212/424] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 03:29:59.334738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1592a7e-21b9-4fde-b4d9-18a5fe48ce0a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:59.346499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4667e7c-0220-4dd6-bbe0-cd38865a134e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:59.356856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:59.388674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcff43-c9f9-48de-9c16-988f95edba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:59.389395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcff43-c9f9-48de-9c16-988f95edba2e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.389807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcff43-c9f9-48de-9c16-988f95edba2e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.390243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcff43-c9f9-48de-9c16-988f95edba2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.402761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcff43-c9f9-48de-9c16-988f95edba2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-b1fcff43-c9f9-48de-9c16-988f95edba2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:59.407998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:59.409086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:59.409972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:59.410826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 213/425] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:59.411997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcff43-c9f9-48de-9c16-988f95edba2e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:59.449190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1592a7e-21b9-4fde-b4d9-18a5fe48ce0a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:59.484895 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352a21f-9b57-4fa8-9cef-eab5291c9688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:59.485815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352a21f-9b57-4fa8-9cef-eab5291c9688 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.486251 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352a21f-9b57-4fa8-9cef-eab5291c9688 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.486674 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352a21f-9b57-4fa8-9cef-eab5291c9688 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.488048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352a21f-9b57-4fa8-9cef-eab5291c9688 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:59.489626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352a21f-9b57-4fa8-9cef-eab5291c9688 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:59.491791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 213/426] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:29:59 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:59.492814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352a21f-9b57-4fa8-9cef-eab5291c9688 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:59.525645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcff43-c9f9-48de-9c16-988f95edba2e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:59.529113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:59.565099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a83352-48a2-48f4-bbaf-26769e3561ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:59.565746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a83352-48a2-48f4-bbaf-26769e3561ce None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.566036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a83352-48a2-48f4-bbaf-26769e3561ce None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.567067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a83352-48a2-48f4-bbaf-26769e3561ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.578774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a83352-48a2-48f4-bbaf-26769e3561ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:29:59.582657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-b2a83352-48a2-48f4-bbaf-26769e3561ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:29:59.585050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:29:59.586199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 03:29:59.587130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:29:59.587759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 214/427] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:29:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:29:59.593608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a83352-48a2-48f4-bbaf-26769e3561ce None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:59.601566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352a21f-9b57-4fa8-9cef-eab5291c9688 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:59.642254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51208699-1bbd-4b93-9e2c-1464da28c61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:59.643142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51208699-1bbd-4b93-9e2c-1464da28c61a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.643592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51208699-1bbd-4b93-9e2c-1464da28c61a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.644029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51208699-1bbd-4b93-9e2c-1464da28c61a None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.644973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51208699-1bbd-4b93-9e2c-1464da28c61a None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:59.645446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51208699-1bbd-4b93-9e2c-1464da28c61a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:59.647113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51208699-1bbd-4b93-9e2c-1464da28c61a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:59.659333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 214/428] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:59.660599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51208699-1bbd-4b93-9e2c-1464da28c61a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:59.715371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a83352-48a2-48f4-bbaf-26769e3561ce None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:59.753880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb7b6-ca63-479a-bf33-0e92340d6d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:59.754587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb7b6-ca63-479a-bf33-0e92340d6d53 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.754871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb7b6-ca63-479a-bf33-0e92340d6d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.755143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb7b6-ca63-479a-bf33-0e92340d6d53 None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.756274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cb7b6-ca63-479a-bf33-0e92340d6d53 None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:59.757716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cb7b6-ca63-479a-bf33-0e92340d6d53 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:59.759498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 215/429] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:29:59 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:59.762573 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb7b6-ca63-479a-bf33-0e92340d6d53 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:59.770132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51208699-1bbd-4b93-9e2c-1464da28c61a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:59.794409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:59.805296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1711af4-66d5-40b1-be18-deaa8e4059b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:59.817597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1711af4-66d5-40b1-be18-deaa8e4059b2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.817597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1711af4-66d5-40b1-be18-deaa8e4059b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.817597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1711af4-66d5-40b1-be18-deaa8e4059b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.817597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1711af4-66d5-40b1-be18-deaa8e4059b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:59.817597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1711af4-66d5-40b1-be18-deaa8e4059b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:59.817597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1711af4-66d5-40b1-be18-deaa8e4059b2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:59.818104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 215/430] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:29:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:59.818870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b93a3-721b-4f72-ac0a-62ce470040cb None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.819152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b93a3-721b-4f72-ac0a-62ce470040cb None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.819425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b93a3-721b-4f72-ac0a-62ce470040cb None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.820633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 216/431] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:59.821328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b93a3-721b-4f72-ac0a-62ce470040cb None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:59.834668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:29:59.874147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb7b6-ca63-479a-bf33-0e92340d6d53 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:59.907570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd4c02-1652-4f0c-967e-8de6fecfdeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:59.908170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4c02-1652-4f0c-967e-8de6fecfdeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.908455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4c02-1652-4f0c-967e-8de6fecfdeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.908733 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4c02-1652-4f0c-967e-8de6fecfdeb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.926341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b93a3-721b-4f72-ac0a-62ce470040cb None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:29:59.959624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900fc60-d391-4779-b884-3b94f452d6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:29:59.960225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900fc60-d391-4779-b884-3b94f452d6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.960501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900fc60-d391-4779-b884-3b94f452d6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.960771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900fc60-d391-4779-b884-3b94f452d6d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.961483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900fc60-d391-4779-b884-3b94f452d6d0 None admin] RBAC: Adding query filter params (group.id=b5477168f623485b9b9c0ad3c9f02b32, role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:29:59.961817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900fc60-d391-4779-b884-3b94f452d6d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:59.963286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900fc60-d391-4779-b884-3b94f452d6d0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:29:59.968825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 217/432] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Sep 17 03:29:59 2022] GET /identity/v3/role_assignments?group.id=b5477168f623485b9b9c0ad3c9f02b32&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:29:59.969984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf897-28fc-4994-a3fb-0c3dd76a1cb5 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:29:59.970270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf897-28fc-4994-a3fb-0c3dd76a1cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:29:59.970543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf897-28fc-4994-a3fb-0c3dd76a1cb5 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:29:59.971895 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 218/433] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:29:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:29:59.972456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcf897-28fc-4994-a3fb-0c3dd76a1cb5 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:29:59.982800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd4c02-1652-4f0c-967e-8de6fecfdeb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:29:59.984286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd4c02-1652-4f0c-967e-8de6fecfdeb4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:00.019781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:00.063031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:00.099009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcf897-28fc-4994-a3fb-0c3dd76a1cb5 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:00.112519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 216/434] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:00.112519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab9de0-ac9a-4d12-ac6c-770ded6672e3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.112519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab9de0-ac9a-4d12-ac6c-770ded6672e3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.112519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab9de0-ac9a-4d12-ac6c-770ded6672e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.131535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-acab9de0-ac9a-4d12-ac6c-770ded6672e3 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:00.132753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-acab9de0-ac9a-4d12-ac6c-770ded6672e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:00.138410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5f42b-4bd4-463e-a2d6-6e53c92a5407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:00.139038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f42b-4bd4-463e-a2d6-6e53c92a5407 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.139318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f42b-4bd4-463e-a2d6-6e53c92a5407 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.139591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f42b-4bd4-463e-a2d6-6e53c92a5407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.213894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 217/435] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:00.215051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff42f8-d937-4fab-a08b-99a2d6ecf241 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.215334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff42f8-d937-4fab-a08b-99a2d6ecf241 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.215609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff42f8-d937-4fab-a08b-99a2d6ecf241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.219432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5f42b-4bd4-463e-a2d6-6e53c92a5407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:00.220888 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5f42b-4bd4-463e-a2d6-6e53c92a5407 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:00.234076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4fff42f8-d937-4fab-a08b-99a2d6ecf241 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:00.235266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4fff42f8-d937-4fab-a08b-99a2d6ecf241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:00.299509 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:00.303993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:00.311361 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 218/436] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:00.312436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff42f8-d937-4fab-a08b-99a2d6ecf241 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:00.342834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 219/437] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:00.348054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5f42b-4bd4-463e-a2d6-6e53c92a5407 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:00.430149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff42f8-d937-4fab-a08b-99a2d6ecf241 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:00.456153 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5f42b-4bd4-463e-a2d6-6e53c92a5407 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:00.474760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537ae52-14d2-4530-b328-7a1eed5c3f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:00.475742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ae52-14d2-4530-b328-7a1eed5c3f82 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.476266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ae52-14d2-4530-b328-7a1eed5c3f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.476873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ae52-14d2-4530-b328-7a1eed5c3f82 None admin] PATH_INFO: `/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.478379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537ae52-14d2-4530-b328-7a1eed5c3f82 None admin] RBAC: Authorizing `identity:get_project(project_id=c8afa64e2aee4a569d1afe94e8ecf1a6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:00.481823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537ae52-14d2-4530-b328-7a1eed5c3f82 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:00.484194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 219/438] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:30:00 2022] GET /identity/v3/projects/c8afa64e2aee4a569d1afe94e8ecf1a6 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:00.489784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537ae52-14d2-4530-b328-7a1eed5c3f82 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:00.502967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b58124-1b7f-4132-be62-13c632ec9ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:00.503600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b58124-1b7f-4132-be62-13c632ec9ffd None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.503880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b58124-1b7f-4132-be62-13c632ec9ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.504153 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b58124-1b7f-4132-be62-13c632ec9ffd None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.505323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b58124-1b7f-4132-be62-13c632ec9ffd None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:00.506803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b58124-1b7f-4132-be62-13c632ec9ffd None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:00.512961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 220/439] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:30:00 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:00.512961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b58124-1b7f-4132-be62-13c632ec9ffd None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:00.557761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:00.559755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:00.604703 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537ae52-14d2-4530-b328-7a1eed5c3f82 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:00.617912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b58124-1b7f-4132-be62-13c632ec9ffd None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:00.639065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef8461-7939-4e96-9715-064085fee215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:00.639695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8461-7939-4e96-9715-064085fee215 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.639977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8461-7939-4e96-9715-064085fee215 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.640256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8461-7939-4e96-9715-064085fee215 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.650225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef8461-7939-4e96-9715-064085fee215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:00.652360 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330fb10-59cd-4fdc-ab3a-7ef35fe3d706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:00.652980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330fb10-59cd-4fdc-ab3a-7ef35fe3d706 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.653261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330fb10-59cd-4fdc-ab3a-7ef35fe3d706 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.653535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330fb10-59cd-4fdc-ab3a-7ef35fe3d706 None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.654687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330fb10-59cd-4fdc-ab3a-7ef35fe3d706 None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-37ef8461-7939-4e96-9715-064085fee215 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:00.655467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:00.656425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:30:00.658517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:00.659144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330fb10-59cd-4fdc-ab3a-7ef35fe3d706 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:00.659215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 220/440] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:30:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:00.659215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef8461-7939-4e96-9715-064085fee215 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:00.662699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 221/441] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:30:00 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:00.663584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330fb10-59cd-4fdc-ab3a-7ef35fe3d706 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:00.756742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef8461-7939-4e96-9715-064085fee215 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:00.762162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330fb10-59cd-4fdc-ab3a-7ef35fe3d706 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:00.773056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:00.782688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:00.799915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e013544-93b6-4664-8d1e-68e9ac36fbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:00.800579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e013544-93b6-4664-8d1e-68e9ac36fbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.800702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807de3-1df8-46a9-9f84-ccd3149657f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:00.801378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e013544-93b6-4664-8d1e-68e9ac36fbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.801378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e013544-93b6-4664-8d1e-68e9ac36fbe9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.801557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807de3-1df8-46a9-9f84-ccd3149657f9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.801557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807de3-1df8-46a9-9f84-ccd3149657f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.801855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807de3-1df8-46a9-9f84-ccd3149657f9 None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.802979 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74807de3-1df8-46a9-9f84-ccd3149657f9 None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:00.804485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74807de3-1df8-46a9-9f84-ccd3149657f9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:00.806222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 221/442] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:30:00 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:00.807172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807de3-1df8-46a9-9f84-ccd3149657f9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:00.811927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e013544-93b6-4664-8d1e-68e9ac36fbe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-4e013544-93b6-4664-8d1e-68e9ac36fbe9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:00.816924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:00.818025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 222/443] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:00.821473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e013544-93b6-4664-8d1e-68e9ac36fbe9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:00.916907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807de3-1df8-46a9-9f84-ccd3149657f9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:00.920350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e013544-93b6-4664-8d1e-68e9ac36fbe9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:00.951290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f194ca3-1667-406e-a6e1-665bb98e1422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:00.951888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f194ca3-1667-406e-a6e1-665bb98e1422 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.952171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f194ca3-1667-406e-a6e1-665bb98e1422 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.952446 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f194ca3-1667-406e-a6e1-665bb98e1422 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.953100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f194ca3-1667-406e-a6e1-665bb98e1422 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:00.953403 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f194ca3-1667-406e-a6e1-665bb98e1422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:00.953537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9d95d-5e38-45a5-975f-33d9072d9b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:00.954156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9d95d-5e38-45a5-975f-33d9072d9b5a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:00.954426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9d95d-5e38-45a5-975f-33d9072d9b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:00.954692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9d95d-5e38-45a5-975f-33d9072d9b5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:00.954791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f194ca3-1667-406e-a6e1-665bb98e1422 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:00.955403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9d95d-5e38-45a5-975f-33d9072d9b5a None admin] RBAC: Adding query filter params (group.id=b5477168f623485b9b9c0ad3c9f02b32, role.id=d801ebea910445a7a6100b35e07630dd, scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:00.955705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9d95d-5e38-45a5-975f-33d9072d9b5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:00.957115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9d95d-5e38-45a5-975f-33d9072d9b5a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:00.962563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 222/444] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:00.962856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 223/445] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Sep 17 03:30:00 2022] GET /identity/v3/role_assignments?group.id=b5477168f623485b9b9c0ad3c9f02b32&scope.project.id=c8afa64e2aee4a569d1afe94e8ecf1a6&role.id=d801ebea910445a7a6100b35e07630dd => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:00.962856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9d95d-5e38-45a5-975f-33d9072d9b5a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:00.978230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f194ca3-1667-406e-a6e1-665bb98e1422 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:01.015102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:01.027589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:01.086192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f194ca3-1667-406e-a6e1-665bb98e1422 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:01.098971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9d95d-5e38-45a5-975f-33d9072d9b5a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:01.130217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025adc59-3041-414c-831c-c3b0ce25a61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:01.131498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025adc59-3041-414c-831c-c3b0ce25a61f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:01.131952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025adc59-3041-414c-831c-c3b0ce25a61f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:01.132398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025adc59-3041-414c-831c-c3b0ce25a61f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:01.133625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025adc59-3041-414c-831c-c3b0ce25a61f None admin] RBAC: Adding query filter params (group.id=b5477168f623485b9b9c0ad3c9f02b32, role.id=d801ebea910445a7a6100b35e07630dd, scope.project.id=e02b0fbedb42442bab7a31b8db209acb) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:01.134156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025adc59-3041-414c-831c-c3b0ce25a61f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:01.136850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025adc59-3041-414c-831c-c3b0ce25a61f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:01.144565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 223/446] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Sep 17 03:30:00 2022] GET /identity/v3/role_assignments?group.id=b5477168f623485b9b9c0ad3c9f02b32&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=d801ebea910445a7a6100b35e07630dd => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:01.150596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025adc59-3041-414c-831c-c3b0ce25a61f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:01.160952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aa110-76cc-4ecb-9de7-97a55ebce03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:01.162758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aa110-76cc-4ecb-9de7-97a55ebce03f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:01.162758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aa110-76cc-4ecb-9de7-97a55ebce03f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:01.162928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aa110-76cc-4ecb-9de7-97a55ebce03f None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:01.163924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6aa110-76cc-4ecb-9de7-97a55ebce03f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:01.164417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6aa110-76cc-4ecb-9de7-97a55ebce03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:01.167428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6aa110-76cc-4ecb-9de7-97a55ebce03f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:01.179391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 224/447] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:01.186555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aa110-76cc-4ecb-9de7-97a55ebce03f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:01.301284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025adc59-3041-414c-831c-c3b0ce25a61f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:01.321144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:01.326556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:01.334704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aa110-76cc-4ecb-9de7-97a55ebce03f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:01.349416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead35d3-cee4-42fe-b014-09ad89be361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:01.350075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead35d3-cee4-42fe-b014-09ad89be361f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:01.350353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead35d3-cee4-42fe-b014-09ad89be361f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:01.350623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead35d3-cee4-42fe-b014-09ad89be361f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:01.376623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4eaa92-020c-466f-8a4e-c21f6d6a2c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:01.377624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eaa92-020c-466f-8a4e-c21f6d6a2c61 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:01.378194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eaa92-020c-466f-8a4e-c21f6d6a2c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:01.378469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eaa92-020c-466f-8a4e-c21f6d6a2c61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:01.443414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead35d3-cee4-42fe-b014-09ad89be361f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:01.445208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead35d3-cee4-42fe-b014-09ad89be361f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:01.488572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4eaa92-020c-466f-8a4e-c21f6d6a2c61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:01.490068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4eaa92-020c-466f-8a4e-c21f6d6a2c61 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:01.566183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 224/448] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:01.571721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead35d3-cee4-42fe-b014-09ad89be361f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:01.587638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:01.599621 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:01.609532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 225/449] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 03:30:01.615438 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4eaa92-020c-466f-8a4e-c21f6d6a2c61 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:01.694911 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead35d3-cee4-42fe-b014-09ad89be361f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:01.731249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4eaa92-020c-466f-8a4e-c21f6d6a2c61 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:01.739539 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20801d1-f36d-4282-9291-77bfe71d928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:01.739539 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20801d1-f36d-4282-9291-77bfe71d928a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:01.740133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20801d1-f36d-4282-9291-77bfe71d928a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:01.740412 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20801d1-f36d-4282-9291-77bfe71d928a None admin] PATH_INFO: `/v3/projects/e02b0fbedb42442bab7a31b8db209acb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:01.741700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20801d1-f36d-4282-9291-77bfe71d928a None admin] RBAC: Authorizing `identity:get_project(project_id=e02b0fbedb42442bab7a31b8db209acb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:01.743258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20801d1-f36d-4282-9291-77bfe71d928a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:01.745816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 225/450] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:30:01 2022] GET /identity/v3/projects/e02b0fbedb42442bab7a31b8db209acb => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:01.751292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20801d1-f36d-4282-9291-77bfe71d928a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:01.772223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75354e02-4efe-47a1-8bf6-696153fd8d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:01.772850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75354e02-4efe-47a1-8bf6-696153fd8d72 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:01.773127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75354e02-4efe-47a1-8bf6-696153fd8d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:01.773400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75354e02-4efe-47a1-8bf6-696153fd8d72 None admin] PATH_INFO: `/v3/projects/545b5824a9314a3a81e4babed059fa06` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:01.774556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75354e02-4efe-47a1-8bf6-696153fd8d72 None admin] RBAC: Authorizing `identity:get_project(project_id=545b5824a9314a3a81e4babed059fa06)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:01.776558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75354e02-4efe-47a1-8bf6-696153fd8d72 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:01.778219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 226/451] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:30:01 2022] GET /identity/v3/projects/545b5824a9314a3a81e4babed059fa06 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:01.783582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75354e02-4efe-47a1-8bf6-696153fd8d72 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:01.836637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:01.848454 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:01.854550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20801d1-f36d-4282-9291-77bfe71d928a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:01.885781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75354e02-4efe-47a1-8bf6-696153fd8d72 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:01.887885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a665ea-62df-4c95-84fe-b42bbdb4c286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:01.888502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a665ea-62df-4c95-84fe-b42bbdb4c286 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:01.888779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a665ea-62df-4c95-84fe-b42bbdb4c286 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:01.889055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a665ea-62df-4c95-84fe-b42bbdb4c286 None admin] PATH_INFO: `/v3/groups/b5477168f623485b9b9c0ad3c9f02b32` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:01.890178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a665ea-62df-4c95-84fe-b42bbdb4c286 None admin] RBAC: Authorizing `identity:get_group(group_id=b5477168f623485b9b9c0ad3c9f02b32)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:01.891594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a665ea-62df-4c95-84fe-b42bbdb4c286 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:01.893147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 226/452] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:30:01 2022] GET /identity/v3/groups/b5477168f623485b9b9c0ad3c9f02b32 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:01.898865 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a665ea-62df-4c95-84fe-b42bbdb4c286 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:01.922219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fa927-6176-4705-9f76-617a224f3c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:01.922838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fa927-6176-4705-9f76-617a224f3c0e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:01.923113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fa927-6176-4705-9f76-617a224f3c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:01.923380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fa927-6176-4705-9f76-617a224f3c0e None admin] PATH_INFO: `/v3/groups/d291e8482c46487a940dc344619d7980` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:01.924479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fa927-6176-4705-9f76-617a224f3c0e None admin] RBAC: Authorizing `identity:get_group(group_id=d291e8482c46487a940dc344619d7980)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:01.925950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fa927-6176-4705-9f76-617a224f3c0e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:01.927507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 227/453] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Sep 17 03:30:01 2022] GET /identity/v3/groups/d291e8482c46487a940dc344619d7980 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:01.933870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fa927-6176-4705-9f76-617a224f3c0e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:02.001927 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a665ea-62df-4c95-84fe-b42bbdb4c286 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:02.032105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fa927-6176-4705-9f76-617a224f3c0e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:02.035273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66d974-a252-4fa1-9ad1-491bd04d5541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:02.035860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66d974-a252-4fa1-9ad1-491bd04d5541 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:02.036137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66d974-a252-4fa1-9ad1-491bd04d5541 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:02.036407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66d974-a252-4fa1-9ad1-491bd04d5541 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:02.037119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66d974-a252-4fa1-9ad1-491bd04d5541 None admin] RBAC: Adding query filter params (group.id=b5477168f623485b9b9c0ad3c9f02b32, role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba, scope.project.id=e02b0fbedb42442bab7a31b8db209acb) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:02.037423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66d974-a252-4fa1-9ad1-491bd04d5541 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:02.038892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66d974-a252-4fa1-9ad1-491bd04d5541 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:02.044574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 227/454] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Sep 17 03:30:01 2022] GET /identity/v3/role_assignments?group.id=b5477168f623485b9b9c0ad3c9f02b32&scope.project.id=e02b0fbedb42442bab7a31b8db209acb&role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:02.067032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:02.072258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce48ae3-70bb-4f59-a1a3-5f4ba4b52215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:02.074068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48ae3-70bb-4f59-a1a3-5f4ba4b52215 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:02.074168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48ae3-70bb-4f59-a1a3-5f4ba4b52215 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:02.074431 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48ae3-70bb-4f59-a1a3-5f4ba4b52215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:02.075178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce48ae3-70bb-4f59-a1a3-5f4ba4b52215 None admin] RBAC: Adding query filter params (group.id=d291e8482c46487a940dc344619d7980, role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=545b5824a9314a3a81e4babed059fa06) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:02.075487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce48ae3-70bb-4f59-a1a3-5f4ba4b52215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:02.076962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce48ae3-70bb-4f59-a1a3-5f4ba4b52215 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:02.083866 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 228/455] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Sep 17 03:30:01 2022] GET /identity/v3/role_assignments?group.id=d291e8482c46487a940dc344619d7980&scope.project.id=545b5824a9314a3a81e4babed059fa06&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:04.739924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3122ec-ec87-4136-b68f-006a8600daae None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:04.740467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3122ec-ec87-4136-b68f-006a8600daae None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:04.740467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3122ec-ec87-4136-b68f-006a8600daae None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:04.742136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 228/456] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:04.747242 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07683-720e-4e58-8870-ffbd361b4757 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:04.747673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07683-720e-4e58-8870-ffbd361b4757 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:04.747844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07683-720e-4e58-8870-ffbd361b4757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:04.772246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d4b07683-720e-4e58-8870-ffbd361b4757 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:04.773462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d4b07683-720e-4e58-8870-ffbd361b4757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:04.863592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 229/457] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:05.131308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ebd88-066c-4893-977b-4226006ba241 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:05.131610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ebd88-066c-4893-977b-4226006ba241 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:05.131820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ebd88-066c-4893-977b-4226006ba241 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:05.133488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 229/458] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:05.139097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8bb3f-6773-4f30-8421-1739fdf03932 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:05.139377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8bb3f-6773-4f30-8421-1739fdf03932 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:05.139649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8bb3f-6773-4f30-8421-1739fdf03932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:05.166913 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6fb8bb3f-6773-4f30-8421-1739fdf03932 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:05.168105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6fb8bb3f-6773-4f30-8421-1739fdf03932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:05.260361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 230/459] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:05.316037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ebd88-066c-4893-977b-4226006ba241 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:05.348585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:05.469452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ebd88-066c-4893-977b-4226006ba241 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:05.492169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d681f-e21c-4072-9a29-210586acf57d None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:05.492458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d681f-e21c-4072-9a29-210586acf57d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:05.492731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d681f-e21c-4072-9a29-210586acf57d None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:05.498485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 231/460] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:05.503564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff5541-7c7f-48c2-8095-fbd4ebd25a8f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:05.504081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff5541-7c7f-48c2-8095-fbd4ebd25a8f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:05.504553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff5541-7c7f-48c2-8095-fbd4ebd25a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:05.516911 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aaef7-61e7-4bcf-b17f-cac8e2a9d216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:05.517558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aaef7-61e7-4bcf-b17f-cac8e2a9d216 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:05.517867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aaef7-61e7-4bcf-b17f-cac8e2a9d216 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:05.518135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aaef7-61e7-4bcf-b17f-cac8e2a9d216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:05.526915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-efff5541-7c7f-48c2-8095-fbd4ebd25a8f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:05.528656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-efff5541-7c7f-48c2-8095-fbd4ebd25a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:05.539066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90aaef7-61e7-4bcf-b17f-cac8e2a9d216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:05.540556 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90aaef7-61e7-4bcf-b17f-cac8e2a9d216 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c90aaef7-61e7-4bcf-b17f-cac8e2a9d216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:05.571143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:05.572132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:05.574318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:05.574318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:05.574318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:05.574318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 230/461] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:05.580564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aaef7-61e7-4bcf-b17f-cac8e2a9d216 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:05.606139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:05.625380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 232/462] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:30:05.626601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff5541-7c7f-48c2-8095-fbd4ebd25a8f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:05.653108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:05.693558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aaef7-61e7-4bcf-b17f-cac8e2a9d216 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:05.743400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c57717-d3cd-4c9a-83bd-00211e06a446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:05.744449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c57717-d3cd-4c9a-83bd-00211e06a446 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:05.744764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c57717-d3cd-4c9a-83bd-00211e06a446 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:05.745422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c57717-d3cd-4c9a-83bd-00211e06a446 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:05.746160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c57717-d3cd-4c9a-83bd-00211e06a446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:05.746470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c57717-d3cd-4c9a-83bd-00211e06a446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:05.747256 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff5541-7c7f-48c2-8095-fbd4ebd25a8f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:05.747893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c57717-d3cd-4c9a-83bd-00211e06a446 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:05.765308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 231/463] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:05.767404 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e989c-4d59-4ffd-82eb-4efca5f4c8c0 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:05.767735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e989c-4d59-4ffd-82eb-4efca5f4c8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:05.768016 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e989c-4d59-4ffd-82eb-4efca5f4c8c0 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:05.769504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 232/464] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:05.771314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e989c-4d59-4ffd-82eb-4efca5f4c8c0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:05.792507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86f987-2cca-4067-b5ce-7866332d3781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:05.793578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86f987-2cca-4067-b5ce-7866332d3781 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:05.793938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86f987-2cca-4067-b5ce-7866332d3781 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:05.794274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86f987-2cca-4067-b5ce-7866332d3781 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:05.817768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86f987-2cca-4067-b5ce-7866332d3781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:05.817768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86f987-2cca-4067-b5ce-7866332d3781 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-1f86f987-2cca-4067-b5ce-7866332d3781 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:05.830954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:05.832044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:05.832935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:05.832935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:05.832935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:05.833666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 233/465] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:05.834724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63a6c0-2759-4340-bcdb-1e8321e7a315 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:05.834997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63a6c0-2759-4340-bcdb-1e8321e7a315 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:05.835311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63a6c0-2759-4340-bcdb-1e8321e7a315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:05.859850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-9f63a6c0-2759-4340-bcdb-1e8321e7a315 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:05.861027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9f63a6c0-2759-4340-bcdb-1e8321e7a315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:05.891219 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e989c-4d59-4ffd-82eb-4efca5f4c8c0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:05.903234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:05.920964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:05.930378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c1eb9-24ff-4ae1-b8bf-d9144751b80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:05.930991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c1eb9-24ff-4ae1-b8bf-d9144751b80b None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:05.931292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c1eb9-24ff-4ae1-b8bf-d9144751b80b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:05.931570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c1eb9-24ff-4ae1-b8bf-d9144751b80b None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:05.932754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c1eb9-24ff-4ae1-b8bf-d9144751b80b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:05.934770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c1eb9-24ff-4ae1-b8bf-d9144751b80b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:05.947276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 234/466] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:05.949048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63a6c0-2759-4340-bcdb-1e8321e7a315 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:06.033161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 233/467] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:30:05 2022] POST /identity/v3/users => generated 246 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:30:06.033161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c1eb9-24ff-4ae1-b8bf-d9144751b80b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:06.084797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63a6c0-2759-4340-bcdb-1e8321e7a315 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:06.153943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67001cd2-95bd-44b6-bdab-58359cfc95ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:06.154562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67001cd2-95bd-44b6-bdab-58359cfc95ec None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:06.154838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67001cd2-95bd-44b6-bdab-58359cfc95ec None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:06.155900 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67001cd2-95bd-44b6-bdab-58359cfc95ec None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:06.156626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67001cd2-95bd-44b6-bdab-58359cfc95ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:06.156930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67001cd2-95bd-44b6-bdab-58359cfc95ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:06.159318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67001cd2-95bd-44b6-bdab-58359cfc95ec None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:06.187325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 235/468] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:06.191761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67001cd2-95bd-44b6-bdab-58359cfc95ec None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:06.239206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c1eb9-24ff-4ae1-b8bf-d9144751b80b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:06.255551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:06.289423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f6e1c-5652-4557-89ef-63bdc2cd91ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:06.290456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f6e1c-5652-4557-89ef-63bdc2cd91ef None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:06.292094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f6e1c-5652-4557-89ef-63bdc2cd91ef None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:06.292441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f6e1c-5652-4557-89ef-63bdc2cd91ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:06.304490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f6e1c-5652-4557-89ef-63bdc2cd91ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:06.306037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f6e1c-5652-4557-89ef-63bdc2cd91ef None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-7e5f6e1c-5652-4557-89ef-63bdc2cd91ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:06.321836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:06.323374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:06.324596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:06.324596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:06.324596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:06.324596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 234/469] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:06.325300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f6e1c-5652-4557-89ef-63bdc2cd91ef None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:06.346239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67001cd2-95bd-44b6-bdab-58359cfc95ec None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:06.365900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:06.390810 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c434ac-4200-4482-a506-7c539591c9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:06.391629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c434ac-4200-4482-a506-7c539591c9ff None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:06.391902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c434ac-4200-4482-a506-7c539591c9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:06.392612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c434ac-4200-4482-a506-7c539591c9ff None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:06.393575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c434ac-4200-4482-a506-7c539591c9ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:06.395319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c434ac-4200-4482-a506-7c539591c9ff None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:06.444118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f6e1c-5652-4557-89ef-63bdc2cd91ef None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:06.475473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 236/470] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:30:06 2022] POST /identity/v3/users => generated 248 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:30:06.476838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c434ac-4200-4482-a506-7c539591c9ff None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:06.496097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d365c1b-7797-4939-b58d-a77a4467bfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:06.496755 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d365c1b-7797-4939-b58d-a77a4467bfc1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:06.497032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d365c1b-7797-4939-b58d-a77a4467bfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:06.497364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d365c1b-7797-4939-b58d-a77a4467bfc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:06.514544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d365c1b-7797-4939-b58d-a77a4467bfc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:06.516160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d365c1b-7797-4939-b58d-a77a4467bfc1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-7d365c1b-7797-4939-b58d-a77a4467bfc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:06.534196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:06.535199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:06.536058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:06.536058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:06.536058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:06.541034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 235/471] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:06.542853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d365c1b-7797-4939-b58d-a77a4467bfc1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:06.604414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:06.622125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c434ac-4200-4482-a506-7c539591c9ff None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:06.661998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d7c8e-1a6a-4a83-8303-70396009c674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:06.662609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d7c8e-1a6a-4a83-8303-70396009c674 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:06.662940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d7c8e-1a6a-4a83-8303-70396009c674 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:06.663145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d7c8e-1a6a-4a83-8303-70396009c674 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:06.663830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d7c8e-1a6a-4a83-8303-70396009c674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:06.664134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d7c8e-1a6a-4a83-8303-70396009c674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:06.665533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d7c8e-1a6a-4a83-8303-70396009c674 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:06.669978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:06.679943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 237/472] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:06.682758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937fe72-be8b-4184-b049-4ce91bd0add6 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:06.683014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937fe72-be8b-4184-b049-4ce91bd0add6 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:06.683285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937fe72-be8b-4184-b049-4ce91bd0add6 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:06.684924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 238/473] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:06.690402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68992fde-bd3c-4d36-8d49-29ab1c4ebb47 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:06.690703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68992fde-bd3c-4d36-8d49-29ab1c4ebb47 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:06.690971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68992fde-bd3c-4d36-8d49-29ab1c4ebb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:06.692256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d365c1b-7797-4939-b58d-a77a4467bfc1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:06.712429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-68992fde-bd3c-4d36-8d49-29ab1c4ebb47 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:06.714546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-68992fde-bd3c-4d36-8d49-29ab1c4ebb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:06.737622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33a76e-f57e-440b-a430-e760d9ea8fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:06.738273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33a76e-f57e-440b-a430-e760d9ea8fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:06.739171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33a76e-f57e-440b-a430-e760d9ea8fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:06.739171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33a76e-f57e-440b-a430-e760d9ea8fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:06.739550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33a76e-f57e-440b-a430-e760d9ea8fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:06.740305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33a76e-f57e-440b-a430-e760d9ea8fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:06.742014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33a76e-f57e-440b-a430-e760d9ea8fa3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:06.759071 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 236/474] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:06.760048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33a76e-f57e-440b-a430-e760d9ea8fa3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:06.798168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 239/475] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:06.801596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68992fde-bd3c-4d36-8d49-29ab1c4ebb47 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:06.880814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33a76e-f57e-440b-a430-e760d9ea8fa3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:06.933481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b49d4a-0fc4-4362-a6c2-b2dc77ea1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:06.934594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b49d4a-0fc4-4362-a6c2-b2dc77ea1d2c None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:06.937193 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b49d4a-0fc4-4362-a6c2-b2dc77ea1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:06.937757 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b49d4a-0fc4-4362-a6c2-b2dc77ea1d2c None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:06.938617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b49d4a-0fc4-4362-a6c2-b2dc77ea1d2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:06.940266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b49d4a-0fc4-4362-a6c2-b2dc77ea1d2c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:06.941612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:06.984547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68992fde-bd3c-4d36-8d49-29ab1c4ebb47 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:07.021421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45eaad-bc51-4ee0-ac62-b661441f41bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:07.026891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45eaad-bc51-4ee0-ac62-b661441f41bd None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:07.026891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45eaad-bc51-4ee0-ac62-b661441f41bd None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:07.026891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45eaad-bc51-4ee0-ac62-b661441f41bd None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:07.026891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45eaad-bc51-4ee0-ac62-b661441f41bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:07.026891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45eaad-bc51-4ee0-ac62-b661441f41bd None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:07.051589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 237/476] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:30:06 2022] POST /identity/v3/users => generated 249 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:30:07.082371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b49d4a-0fc4-4362-a6c2-b2dc77ea1d2c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:07.085464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:07.113592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 240/477] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:30:06 2022] POST /identity/v3/users => generated 248 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:30:07.242292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b49d4a-0fc4-4362-a6c2-b2dc77ea1d2c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:07.296839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd2783-ec66-45b0-8476-1d10589ea816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:07.297472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd2783-ec66-45b0-8476-1d10589ea816 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:07.297793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd2783-ec66-45b0-8476-1d10589ea816 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:07.298072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd2783-ec66-45b0-8476-1d10589ea816 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:07.298760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd2783-ec66-45b0-8476-1d10589ea816 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:07.300149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd2783-ec66-45b0-8476-1d10589ea816 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:07.345436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.sql.core [None req-06dd2783-ec66-45b0-8476-1d10589ea816 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 03:30:07.345436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 17 03:30:07.345436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [parameters: {'id': '23b6630cb64c4d0f8646410b0644768d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 17 03:30:07.345436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81725) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-06dd2783-ec66-45b0-8476-1d10589ea816 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application conn.query(q) Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result.read() Sep 17 03:30:07.362499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return ref.to_dict() Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 03:30:07.365637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application next(self.gen) Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application yield resource Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application next(self.gen) Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise self.value Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application session.commit() Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 03:30:07.366701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.session.flush() Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application compat.raise_( Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 03:30:07.367590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 03:30:07.368479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application conn.query(q) Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result.read() Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 03:30:07.369383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [parameters: {'id': '23b6630cb64c4d0f8646410b0644768d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:07.371511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 17 03:30:07.373419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:07.377906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 238/478] 10.222.0.42 () {60 vars in 1131 bytes} [Sat Sep 17 03:30:07 2022] POST /identity/v3/roles => generated 144 bytes in 282 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:07.379561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45eaad-bc51-4ee0-ac62-b661441f41bd None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:07.500233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45eaad-bc51-4ee0-ac62-b661441f41bd None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:07.534771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:07.540870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09b9ef-e0bd-4731-a97a-c9cdd7f4e521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:07.541584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b9ef-e0bd-4731-a97a-c9cdd7f4e521 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:07.541947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b9ef-e0bd-4731-a97a-c9cdd7f4e521 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:07.542229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b9ef-e0bd-4731-a97a-c9cdd7f4e521 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:07.553571 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09b9ef-e0bd-4731-a97a-c9cdd7f4e521 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-5c09b9ef-e0bd-4731-a97a-c9cdd7f4e521 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:07.559025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:07.559915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:07.560745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 241/479] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:30:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:07.567767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd2783-ec66-45b0-8476-1d10589ea816 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:07.677104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:07.703072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd2783-ec66-45b0-8476-1d10589ea816 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:07.746448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de401f-8423-4c71-8034-47f336255dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:07.747077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de401f-8423-4c71-8034-47f336255dff None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:07.747364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de401f-8423-4c71-8034-47f336255dff None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:07.747640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de401f-8423-4c71-8034-47f336255dff None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:07.748310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de401f-8423-4c71-8034-47f336255dff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:07.748616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de401f-8423-4c71-8034-47f336255dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:07.750022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de401f-8423-4c71-8034-47f336255dff None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:07.759685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 239/480] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:08.755290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e193e0-bd27-4442-a847-0b0aef401e5a None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:08.755911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e193e0-bd27-4442-a847-0b0aef401e5a None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:08.755911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e193e0-bd27-4442-a847-0b0aef401e5a None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:08.757538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 242/481] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:08.765437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d920af0-adca-4794-a1f4-7afd0d0b69b2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:08.765437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d920af0-adca-4794-a1f4-7afd0d0b69b2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:08.765437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d920af0-adca-4794-a1f4-7afd0d0b69b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:08.790694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3d920af0-adca-4794-a1f4-7afd0d0b69b2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:08.792467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3d920af0-adca-4794-a1f4-7afd0d0b69b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:08.876784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 240/482] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:09.050559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b08b6-9f83-4eac-af60-ccda3eb32682 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:09.051241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b08b6-9f83-4eac-af60-ccda3eb32682 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:09.051659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b08b6-9f83-4eac-af60-ccda3eb32682 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:09.053401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 243/483] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:09.058929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf95b7-a24e-435f-b94a-5539d39a2c46 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:09.059329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf95b7-a24e-435f-b94a-5539d39a2c46 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:09.059665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf95b7-a24e-435f-b94a-5539d39a2c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:09.084894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-50bf95b7-a24e-435f-b94a-5539d39a2c46 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:09.086108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-50bf95b7-a24e-435f-b94a-5539d39a2c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:09.148644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:09.167865 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 241/484] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:09.321896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b08b6-9f83-4eac-af60-ccda3eb32682 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:09.447956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b08b6-9f83-4eac-af60-ccda3eb32682 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:09.530677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ca285-e575-45ce-b736-ec2be1cb4445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:09.531701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ca285-e575-45ce-b736-ec2be1cb4445 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:09.532213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ca285-e575-45ce-b736-ec2be1cb4445 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:09.532680 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ca285-e575-45ce-b736-ec2be1cb4445 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:09.543512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ca285-e575-45ce-b736-ec2be1cb4445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:09.545972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf95b7-a24e-435f-b94a-5539d39a2c46 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-fc9ca285-e575-45ce-b736-ec2be1cb4445 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:09.550125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:09.551009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:09.551860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:09.555414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 244/485] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:09.561857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ca285-e575-45ce-b736-ec2be1cb4445 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:09.651472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:09.702670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf95b7-a24e-435f-b94a-5539d39a2c46 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:09.743697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ca285-e575-45ce-b736-ec2be1cb4445 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:09.748395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8359d27-cf0d-46b5-9963-aada98818ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:09.749030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8359d27-cf0d-46b5-9963-aada98818ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:09.749313 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8359d27-cf0d-46b5-9963-aada98818ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:09.749588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8359d27-cf0d-46b5-9963-aada98818ee7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:09.767862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8359d27-cf0d-46b5-9963-aada98818ee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a8359d27-cf0d-46b5-9963-aada98818ee7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:09.781012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:09.783447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:09.784274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 242/486] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:09.792159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8359d27-cf0d-46b5-9963-aada98818ee7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:09.797757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8a45-a7a6-46f7-9085-012fc8b839e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:09.797757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8a45-a7a6-46f7-9085-012fc8b839e0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:09.797757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8a45-a7a6-46f7-9085-012fc8b839e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:09.797757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8a45-a7a6-46f7-9085-012fc8b839e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:09.798485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c8a45-a7a6-46f7-9085-012fc8b839e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:09.798485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c8a45-a7a6-46f7-9085-012fc8b839e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:09.806071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c8a45-a7a6-46f7-9085-012fc8b839e0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:09.825068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 245/487] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:09.839534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8a45-a7a6-46f7-9085-012fc8b839e0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:09.860986 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:09.926547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8359d27-cf0d-46b5-9963-aada98818ee7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:09.968902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf3e76-e91b-4711-bcdf-6fc07842f3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:09.969928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf3e76-e91b-4711-bcdf-6fc07842f3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:09.970210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf3e76-e91b-4711-bcdf-6fc07842f3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:09.970485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf3e76-e91b-4711-bcdf-6fc07842f3b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:09.971421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf3e76-e91b-4711-bcdf-6fc07842f3b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:09.971725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf3e76-e91b-4711-bcdf-6fc07842f3b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:09.973825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf3e76-e91b-4711-bcdf-6fc07842f3b5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:09.985496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 243/488] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:09.987116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5c32c-bf0f-45a5-88e8-c6d9507ecf42 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:09.987655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5c32c-bf0f-45a5-88e8-c6d9507ecf42 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:09.987932 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5c32c-bf0f-45a5-88e8-c6d9507ecf42 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:09.993800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 244/489] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:09.995623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5c32c-bf0f-45a5-88e8-c6d9507ecf42 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:09.999231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8a45-a7a6-46f7-9085-012fc8b839e0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:10.014811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:10.036493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f96673-b623-47e5-84a3-e7a22f741529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:10.037120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f96673-b623-47e5-84a3-e7a22f741529 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.037392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f96673-b623-47e5-84a3-e7a22f741529 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.037659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f96673-b623-47e5-84a3-e7a22f741529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.050785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f96673-b623-47e5-84a3-e7a22f741529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:10.052279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f96673-b623-47e5-84a3-e7a22f741529 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-24f96673-b623-47e5-84a3-e7a22f741529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:10.067265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:10.068190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:10.069644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:10.069644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:10.069644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:10.069644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 246/490] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:10.069644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addc66c-9cc4-44f1-82d3-fad6bf557af7 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.069961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addc66c-9cc4-44f1-82d3-fad6bf557af7 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.070027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addc66c-9cc4-44f1-82d3-fad6bf557af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.089305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-2addc66c-9cc4-44f1-82d3-fad6bf557af7 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:10.090525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-2addc66c-9cc4-44f1-82d3-fad6bf557af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:10.114282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5c32c-bf0f-45a5-88e8-c6d9507ecf42 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:10.126578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:10.156937 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901044c-1398-49b8-9d54-38dfafab31f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:10.157427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 247/491] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:10.158165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901044c-1398-49b8-9d54-38dfafab31f2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.158650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901044c-1398-49b8-9d54-38dfafab31f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.158755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0b1d-4d75-4971-993a-419b9be15810 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.159036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0b1d-4d75-4971-993a-419b9be15810 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.159305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0b1d-4d75-4971-993a-419b9be15810 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.159646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901044c-1398-49b8-9d54-38dfafab31f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.160817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 248/492] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:10.161510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc0b1d-4d75-4971-993a-419b9be15810 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:10.177769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901044c-1398-49b8-9d54-38dfafab31f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:10.179850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901044c-1398-49b8-9d54-38dfafab31f2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a901044c-1398-49b8-9d54-38dfafab31f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:10.196045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:10.197105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:10.198033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:10.198033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:10.198033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:10.202089 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 245/493] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:10.203418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67098-465d-4f40-bbc1-5e004a81bc51 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.203703 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67098-465d-4f40-bbc1-5e004a81bc51 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.203978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67098-465d-4f40-bbc1-5e004a81bc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.223765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-46c67098-465d-4f40-bbc1-5e004a81bc51 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:10.224992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-46c67098-465d-4f40-bbc1-5e004a81bc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:10.281485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:10.289327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc0b1d-4d75-4971-993a-419b9be15810 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:10.306488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 246/494] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:10.309042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c67098-465d-4f40-bbc1-5e004a81bc51 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:10.351654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d3bad-87ca-4a34-8f75-196065f3b20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:10.352257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d3bad-87ca-4a34-8f75-196065f3b20c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.352530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d3bad-87ca-4a34-8f75-196065f3b20c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.352796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d3bad-87ca-4a34-8f75-196065f3b20c None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.353436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d3bad-87ca-4a34-8f75-196065f3b20c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:10.353805 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d3bad-87ca-4a34-8f75-196065f3b20c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:10.358523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d3bad-87ca-4a34-8f75-196065f3b20c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:10.383243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 249/495] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:10.389015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d3bad-87ca-4a34-8f75-196065f3b20c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:10.482821 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:10.500017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c67098-465d-4f40-bbc1-5e004a81bc51 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:10.519565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d3bad-87ca-4a34-8f75-196065f3b20c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:10.545941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d571a2-6f47-453e-87e3-1628a3014dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:10.546613 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d571a2-6f47-453e-87e3-1628a3014dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.547156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d571a2-6f47-453e-87e3-1628a3014dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.547437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d571a2-6f47-453e-87e3-1628a3014dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.548501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d571a2-6f47-453e-87e3-1628a3014dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:10.548928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d571a2-6f47-453e-87e3-1628a3014dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:10.550743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d571a2-6f47-453e-87e3-1628a3014dd7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:10.555932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab112dd-3e24-400c-aaf0-f4337755158e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:10.556657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab112dd-3e24-400c-aaf0-f4337755158e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.557581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab112dd-3e24-400c-aaf0-f4337755158e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.557967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab112dd-3e24-400c-aaf0-f4337755158e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.564822 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 247/496] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:10.565663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d571a2-6f47-453e-87e3-1628a3014dd7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:10.575187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab112dd-3e24-400c-aaf0-f4337755158e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:10.577447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab112dd-3e24-400c-aaf0-f4337755158e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-dab112dd-3e24-400c-aaf0-f4337755158e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:10.594340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:10.595357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:10.596276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 250/497] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Sep 17 03:30:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:10.596879 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab112dd-3e24-400c-aaf0-f4337755158e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:10.615175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:10.685375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d571a2-6f47-453e-87e3-1628a3014dd7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:10.709659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab112dd-3e24-400c-aaf0-f4337755158e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:10.721084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b40506-236d-491a-ad8d-e7269396470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:10.721706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b40506-236d-491a-ad8d-e7269396470b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.722010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b40506-236d-491a-ad8d-e7269396470b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.722282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b40506-236d-491a-ad8d-e7269396470b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.732545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b40506-236d-491a-ad8d-e7269396470b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-86b40506-236d-491a-ad8d-e7269396470b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:10.737776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:10.738701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:10.739500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 248/498] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:10.740717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b40506-236d-491a-ad8d-e7269396470b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:10.745860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248e03a-736c-446a-9077-022dabda1299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:10.746482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248e03a-736c-446a-9077-022dabda1299 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.746755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248e03a-736c-446a-9077-022dabda1299 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.747021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248e03a-736c-446a-9077-022dabda1299 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.753847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:10.754764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248e03a-736c-446a-9077-022dabda1299 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:10.756224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248e03a-736c-446a-9077-022dabda1299 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-7248e03a-736c-446a-9077-022dabda1299 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:10.767970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:10.768905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 251/499] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:10.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248e03a-736c-446a-9077-022dabda1299 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:10.849195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b40506-236d-491a-ad8d-e7269396470b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:10.862049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:10.883063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409611c-70fa-49bb-a1c5-2f31839f2775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:10.883752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409611c-70fa-49bb-a1c5-2f31839f2775 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.884027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409611c-70fa-49bb-a1c5-2f31839f2775 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.884299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409611c-70fa-49bb-a1c5-2f31839f2775 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.887302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248e03a-736c-446a-9077-022dabda1299 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:10.895016 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409611c-70fa-49bb-a1c5-2f31839f2775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4409611c-70fa-49bb-a1c5-2f31839f2775 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:10.899688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:10.901199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:10.902308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 249/500] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:10.902981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6fd3e-268c-4368-91c5-336146f1a1f1 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.902981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6fd3e-268c-4368-91c5-336146f1a1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.903596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6fd3e-268c-4368-91c5-336146f1a1f1 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.904891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 250/501] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:10.905579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6fd3e-268c-4368-91c5-336146f1a1f1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:10.925289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e57fb-217c-40a4-84f6-bc7a5da1f7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:10.925942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e57fb-217c-40a4-84f6-bc7a5da1f7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:10.926221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e57fb-217c-40a4-84f6-bc7a5da1f7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:10.926494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e57fb-217c-40a4-84f6-bc7a5da1f7f8 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:10.927223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e57fb-217c-40a4-84f6-bc7a5da1f7f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:10.927529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e57fb-217c-40a4-84f6-bc7a5da1f7f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:10.928981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e57fb-217c-40a4-84f6-bc7a5da1f7f8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:10.965745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 252/502] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Sep 17 03:30:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:10.966819 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e57fb-217c-40a4-84f6-bc7a5da1f7f8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:10.981286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:11.010975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6fd3e-268c-4368-91c5-336146f1a1f1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:11.045784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4e9ed-e096-47cf-865d-71405ee86bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:11.046383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4e9ed-e096-47cf-865d-71405ee86bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:11.046666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4e9ed-e096-47cf-865d-71405ee86bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:11.046938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4e9ed-e096-47cf-865d-71405ee86bb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:11.047586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4e9ed-e096-47cf-865d-71405ee86bb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:11.047887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4e9ed-e096-47cf-865d-71405ee86bb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:11.049263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4e9ed-e096-47cf-865d-71405ee86bb3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:11.057044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 251/503] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:11.058204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a765bf-dcb2-4146-a76c-e449e37bed94 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:11.058489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a765bf-dcb2-4146-a76c-e449e37bed94 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:11.058763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a765bf-dcb2-4146-a76c-e449e37bed94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:11.065171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e57fb-217c-40a4-84f6-bc7a5da1f7f8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:11.076669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-70a765bf-dcb2-4146-a76c-e449e37bed94 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:11.077880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-70a765bf-dcb2-4146-a76c-e449e37bed94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:11.095774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:11.101095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ee73c-3e6c-4b8e-854a-b8dae3a4b7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:11.101706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ee73c-3e6c-4b8e-854a-b8dae3a4b7da None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:11.102017 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ee73c-3e6c-4b8e-854a-b8dae3a4b7da None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:11.102292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ee73c-3e6c-4b8e-854a-b8dae3a4b7da None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:11.103014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ee73c-3e6c-4b8e-854a-b8dae3a4b7da None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:11.103322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ee73c-3e6c-4b8e-854a-b8dae3a4b7da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:11.105710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ee73c-3e6c-4b8e-854a-b8dae3a4b7da None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:11.122439 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 253/504] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Sep 17 03:30:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:11.123245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ee73c-3e6c-4b8e-854a-b8dae3a4b7da None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:11.150248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 252/505] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:11.151627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a765bf-dcb2-4146-a76c-e449e37bed94 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:11.223499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:11.249565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ee73c-3e6c-4b8e-854a-b8dae3a4b7da None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:11.270260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a765bf-dcb2-4146-a76c-e449e37bed94 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:11.286526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3b2e-d9b6-4d63-a12b-820735e62422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:11.287131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3b2e-d9b6-4d63-a12b-820735e62422 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:11.287410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3b2e-d9b6-4d63-a12b-820735e62422 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:11.287686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3b2e-d9b6-4d63-a12b-820735e62422 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:11.288343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d3b2e-d9b6-4d63-a12b-820735e62422 None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:11.288642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d3b2e-d9b6-4d63-a12b-820735e62422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:11.290159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d3b2e-d9b6-4d63-a12b-820735e62422 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:11.299787 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 254/506] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:11.301067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3b2e-d9b6-4d63-a12b-820735e62422 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:11.316266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf21824-3b39-44b3-8f26-e4cd5e3b44e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:11.317016 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf21824-3b39-44b3-8f26-e4cd5e3b44e4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:11.317398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf21824-3b39-44b3-8f26-e4cd5e3b44e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:11.317812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf21824-3b39-44b3-8f26-e4cd5e3b44e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:11.358121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:11.389947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf21824-3b39-44b3-8f26-e4cd5e3b44e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:11.391672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf21824-3b39-44b3-8f26-e4cd5e3b44e4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:11.409063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3b2e-d9b6-4d63-a12b-820735e62422 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:11.444064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1111db0-273a-4883-b991-3d6c66e61846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:11.444771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1111db0-273a-4883-b991-3d6c66e61846 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:11.445118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1111db0-273a-4883-b991-3d6c66e61846 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:11.445385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1111db0-273a-4883-b991-3d6c66e61846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:11.455167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:11.456752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1111db0-273a-4883-b991-3d6c66e61846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:11.458281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1111db0-273a-4883-b991-3d6c66e61846 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-a1111db0-273a-4883-b991-3d6c66e61846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:11.474088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:11.481337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:11.482426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:11.482426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:11.482426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:11.482426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 255/507] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:11.482426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1111db0-273a-4883-b991-3d6c66e61846 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:11.541082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 253/508] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:11.542040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf21824-3b39-44b3-8f26-e4cd5e3b44e4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:11.578138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1111db0-273a-4883-b991-3d6c66e61846 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:11.580414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:11.611778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1486b-6c67-4169-86cc-ccf948e26d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:11.612380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1486b-6c67-4169-86cc-ccf948e26d0f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:11.612661 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1486b-6c67-4169-86cc-ccf948e26d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:11.612933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1486b-6c67-4169-86cc-ccf948e26d0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:11.641287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf21824-3b39-44b3-8f26-e4cd5e3b44e4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:11.674128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca597f20-129e-40ff-a1d7-e7194a91aeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:11.674733 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca597f20-129e-40ff-a1d7-e7194a91aeb6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:11.675015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca597f20-129e-40ff-a1d7-e7194a91aeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:11.675282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca597f20-129e-40ff-a1d7-e7194a91aeb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:11.680335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1486b-6c67-4169-86cc-ccf948e26d0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:11.682507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1486b-6c67-4169-86cc-ccf948e26d0f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:11.685607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca597f20-129e-40ff-a1d7-e7194a91aeb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:11.687053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca597f20-129e-40ff-a1d7-e7194a91aeb6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ca597f20-129e-40ff-a1d7-e7194a91aeb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:11.701466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:11.702594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:11.703507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:11.703507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:11.703507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:11.705864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 254/509] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:11.707192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca597f20-129e-40ff-a1d7-e7194a91aeb6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:11.726064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:11.787793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 256/510] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:11.788643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1486b-6c67-4169-86cc-ccf948e26d0f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:11.791870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:11.814962 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca597f20-129e-40ff-a1d7-e7194a91aeb6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:11.849531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c637fb-fe8a-4cea-9797-ec1cd8a7bc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:11.850178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c637fb-fe8a-4cea-9797-ec1cd8a7bc55 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:11.850443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c637fb-fe8a-4cea-9797-ec1cd8a7bc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:11.850717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c637fb-fe8a-4cea-9797-ec1cd8a7bc55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:11.861120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c637fb-fe8a-4cea-9797-ec1cd8a7bc55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:11.862576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c637fb-fe8a-4cea-9797-ec1cd8a7bc55 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e6c637fb-fe8a-4cea-9797-ec1cd8a7bc55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:11.877363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:11.878430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:11.879311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:11.879311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:11.879311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:11.879311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 255/511] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:11.880600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c637fb-fe8a-4cea-9797-ec1cd8a7bc55 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:11.888771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1486b-6c67-4169-86cc-ccf948e26d0f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:11.921814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91963ccf-dadb-42bf-97ef-e8af416c7dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:11.922420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91963ccf-dadb-42bf-97ef-e8af416c7dea None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:11.922699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91963ccf-dadb-42bf-97ef-e8af416c7dea None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:11.922974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91963ccf-dadb-42bf-97ef-e8af416c7dea None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:11.923614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91963ccf-dadb-42bf-97ef-e8af416c7dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:11.923915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91963ccf-dadb-42bf-97ef-e8af416c7dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:11.925270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91963ccf-dadb-42bf-97ef-e8af416c7dea None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:11.937029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 257/512] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:11.937955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91963ccf-dadb-42bf-97ef-e8af416c7dea None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:11.946533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:11.980627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c637fb-fe8a-4cea-9797-ec1cd8a7bc55 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.002977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:12.013694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583580-3d2f-46f3-9318-8e0734a9821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.014351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583580-3d2f-46f3-9318-8e0734a9821b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.014628 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583580-3d2f-46f3-9318-8e0734a9821b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.014951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583580-3d2f-46f3-9318-8e0734a9821b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.026183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58583580-3d2f-46f3-9318-8e0734a9821b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.027646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58583580-3d2f-46f3-9318-8e0734a9821b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.040412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91963ccf-dadb-42bf-97ef-e8af416c7dea None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-58583580-3d2f-46f3-9318-8e0734a9821b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:12.042046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:12.043067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:12.043924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:12.043924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:12.043924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:12.043924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 256/513] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:12.043924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583580-3d2f-46f3-9318-8e0734a9821b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.083765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88646da2-fc4f-4e9b-ab31-7f930fa1e418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.084366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88646da2-fc4f-4e9b-ab31-7f930fa1e418 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.084636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88646da2-fc4f-4e9b-ab31-7f930fa1e418 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.084901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88646da2-fc4f-4e9b-ab31-7f930fa1e418 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.085532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88646da2-fc4f-4e9b-ab31-7f930fa1e418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:12.085862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88646da2-fc4f-4e9b-ab31-7f930fa1e418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.087226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88646da2-fc4f-4e9b-ab31-7f930fa1e418 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.099626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 258/514] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:12.100419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88646da2-fc4f-4e9b-ab31-7f930fa1e418 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.144532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583580-3d2f-46f3-9318-8e0734a9821b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.164681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:12.178658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6403cf-4053-4fe8-8df0-452c0a341a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.179269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6403cf-4053-4fe8-8df0-452c0a341a1b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.179550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6403cf-4053-4fe8-8df0-452c0a341a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.179824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6403cf-4053-4fe8-8df0-452c0a341a1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.190951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6403cf-4053-4fe8-8df0-452c0a341a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.192389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6403cf-4053-4fe8-8df0-452c0a341a1b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.196085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88646da2-fc4f-4e9b-ab31-7f930fa1e418 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ca6403cf-4053-4fe8-8df0-452c0a341a1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:12.207240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:12.208155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:12.209028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:12.209028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:12.209028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:12.209028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 257/515] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:12.209254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6403cf-4053-4fe8-8df0-452c0a341a1b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.223689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:12.235004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf079ba1-741a-4f9a-b91d-6c01aa76249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.235639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079ba1-741a-4f9a-b91d-6c01aa76249d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.235988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079ba1-741a-4f9a-b91d-6c01aa76249d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.236258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079ba1-741a-4f9a-b91d-6c01aa76249d None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.236890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf079ba1-741a-4f9a-b91d-6c01aa76249d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:12.237190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf079ba1-741a-4f9a-b91d-6c01aa76249d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.238591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf079ba1-741a-4f9a-b91d-6c01aa76249d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.250470 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 259/516] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:12.251279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf079ba1-741a-4f9a-b91d-6c01aa76249d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.312407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6403cf-4053-4fe8-8df0-452c0a341a1b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.349666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbaddc9-dc94-4296-bed5-ecc95ad4479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.350315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaddc9-dc94-4296-bed5-ecc95ad4479b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.350597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaddc9-dc94-4296-bed5-ecc95ad4479b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.350869 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaddc9-dc94-4296-bed5-ecc95ad4479b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.352383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf079ba1-741a-4f9a-b91d-6c01aa76249d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.358895 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbaddc9-dc94-4296-bed5-ecc95ad4479b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.360328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbaddc9-dc94-4296-bed5-ecc95ad4479b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-dfbaddc9-dc94-4296-bed5-ecc95ad4479b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:12.372656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:12.373610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 258/517] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:12.374560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbaddc9-dc94-4296-bed5-ecc95ad4479b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.389930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8781a10-cce3-4b9b-b5f1-58802004d3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.390530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8781a10-cce3-4b9b-b5f1-58802004d3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.390802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8781a10-cce3-4b9b-b5f1-58802004d3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.391066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8781a10-cce3-4b9b-b5f1-58802004d3b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.391695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8781a10-cce3-4b9b-b5f1-58802004d3b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:12.391990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8781a10-cce3-4b9b-b5f1-58802004d3b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.393340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8781a10-cce3-4b9b-b5f1-58802004d3b0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.404723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 260/518] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:12.405495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8781a10-cce3-4b9b-b5f1-58802004d3b0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.405613 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:12.452156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:12.475480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbaddc9-dc94-4296-bed5-ecc95ad4479b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.504616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8781a10-cce3-4b9b-b5f1-58802004d3b0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.508860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec9265-0b47-4e49-b644-bd69b362d551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.509489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec9265-0b47-4e49-b644-bd69b362d551 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.509884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec9265-0b47-4e49-b644-bd69b362d551 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.510405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec9265-0b47-4e49-b644-bd69b362d551 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.517519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec9265-0b47-4e49-b644-bd69b362d551 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.519009 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec9265-0b47-4e49-b644-bd69b362d551 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d6ec9265-0b47-4e49-b644-bd69b362d551 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:12.530800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:12.531737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 259/519] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:12.532656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec9265-0b47-4e49-b644-bd69b362d551 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.541967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab7220-e811-4bb0-89ab-a8629d47ab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.542562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab7220-e811-4bb0-89ab-a8629d47ab6a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.542834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab7220-e811-4bb0-89ab-a8629d47ab6a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.543108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab7220-e811-4bb0-89ab-a8629d47ab6a None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.543736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab7220-e811-4bb0-89ab-a8629d47ab6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:12.544034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab7220-e811-4bb0-89ab-a8629d47ab6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.545380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab7220-e811-4bb0-89ab-a8629d47ab6a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.556535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 261/520] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:12.557998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab7220-e811-4bb0-89ab-a8629d47ab6a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.624306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:12.632357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec9265-0b47-4e49-b644-bd69b362d551 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.657772 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab7220-e811-4bb0-89ab-a8629d47ab6a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.660074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:12.665907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55e998-e1c2-488d-8c3b-0494e50b0e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.667145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55e998-e1c2-488d-8c3b-0494e50b0e86 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.672144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55e998-e1c2-488d-8c3b-0494e50b0e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.672144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55e998-e1c2-488d-8c3b-0494e50b0e86 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.672144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55e998-e1c2-488d-8c3b-0494e50b0e86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.672144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55e998-e1c2-488d-8c3b-0494e50b0e86 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.690574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56272c91-1b8e-4858-b721-c96bcb51b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.691173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56272c91-1b8e-4858-b721-c96bcb51b342 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.691445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56272c91-1b8e-4858-b721-c96bcb51b342 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.691710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56272c91-1b8e-4858-b721-c96bcb51b342 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.692442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56272c91-1b8e-4858-b721-c96bcb51b342 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:12.692790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56272c91-1b8e-4858-b721-c96bcb51b342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.694527 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56272c91-1b8e-4858-b721-c96bcb51b342 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.711542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 262/521] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:12.712311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56272c91-1b8e-4858-b721-c96bcb51b342 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.767372 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 260/522] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:30:12 2022] POST /identity/v3/users => generated 247 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:30:12.768105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55e998-e1c2-488d-8c3b-0494e50b0e86 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.831131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56272c91-1b8e-4858-b721-c96bcb51b342 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.870839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26345d2-9f00-45de-acb4-60f78e57e200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.871546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26345d2-9f00-45de-acb4-60f78e57e200 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.871810 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26345d2-9f00-45de-acb4-60f78e57e200 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.872077 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26345d2-9f00-45de-acb4-60f78e57e200 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.879527 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26345d2-9f00-45de-acb4-60f78e57e200 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.880925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26345d2-9f00-45de-acb4-60f78e57e200 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.888846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55e998-e1c2-488d-8c3b-0494e50b0e86 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-a26345d2-9f00-45de-acb4-60f78e57e200 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:12.894132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:12.895385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:12.896344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:12.896344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:12.896344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:12.896344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:12.896344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:12.896344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:12.896344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:12.896344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:12.896344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 263/523] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:12.896820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26345d2-9f00-45de-acb4-60f78e57e200 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.923444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:12.931768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb7a75-6cd2-4e62-938e-6385358fb798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:12.932763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb7a75-6cd2-4e62-938e-6385358fb798 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:12.932871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb7a75-6cd2-4e62-938e-6385358fb798 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:12.933093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb7a75-6cd2-4e62-938e-6385358fb798 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:12.933887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb7a75-6cd2-4e62-938e-6385358fb798 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:12.939176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb7a75-6cd2-4e62-938e-6385358fb798 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:12.939176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb7a75-6cd2-4e62-938e-6385358fb798 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:12.980560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 261/524] 10.222.0.42 () {56 vars in 1139 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:12.980560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb7a75-6cd2-4e62-938e-6385358fb798 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:12.987888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:13.012085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26345d2-9f00-45de-acb4-60f78e57e200 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:13.047610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5339c-afa6-4aa0-8ceb-37b2188c8f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:13.048227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5339c-afa6-4aa0-8ceb-37b2188c8f37 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:13.048499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5339c-afa6-4aa0-8ceb-37b2188c8f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:13.048762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5339c-afa6-4aa0-8ceb-37b2188c8f37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:13.056248 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5339c-afa6-4aa0-8ceb-37b2188c8f37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:13.057868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5339c-afa6-4aa0-8ceb-37b2188c8f37 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-16f5339c-afa6-4aa0-8ceb-37b2188c8f37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:13.069287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:13.070177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:13.071132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:13.071132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:13.071132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:13.071132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:13.071132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:13.071132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:13.071132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:13.071132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:13.071132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 264/525] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:13.071602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5339c-afa6-4aa0-8ceb-37b2188c8f37 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:13.096155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb7a75-6cd2-4e62-938e-6385358fb798 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:13.131698 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d32f79-1213-4fdf-b828-d1399d432ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:13.132555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d32f79-1213-4fdf-b828-d1399d432ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:13.132849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d32f79-1213-4fdf-b828-d1399d432ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:13.133113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d32f79-1213-4fdf-b828-d1399d432ea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:13.160288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:13.178830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5339c-afa6-4aa0-8ceb-37b2188c8f37 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:13.208635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d32f79-1213-4fdf-b828-d1399d432ea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:13.210605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d32f79-1213-4fdf-b828-d1399d432ea5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:13.216238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e6415-b918-489a-b8b7-2ea0b26f02b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:13.216852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e6415-b918-489a-b8b7-2ea0b26f02b0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:13.217121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e6415-b918-489a-b8b7-2ea0b26f02b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:13.217395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e6415-b918-489a-b8b7-2ea0b26f02b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:13.218115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e6415-b918-489a-b8b7-2ea0b26f02b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:13.218423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e6415-b918-489a-b8b7-2ea0b26f02b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:13.219849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e6415-b918-489a-b8b7-2ea0b26f02b0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:13.219982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:13.238443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 265/526] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:13.239284 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e6415-b918-489a-b8b7-2ea0b26f02b0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:13.326252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 262/527] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:13.327105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d32f79-1213-4fdf-b828-d1399d432ea5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:13.355336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e6415-b918-489a-b8b7-2ea0b26f02b0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:13.395990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6174c0-ae61-44bf-9eb0-97a72d05c434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:13.396842 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6174c0-ae61-44bf-9eb0-97a72d05c434 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:13.397115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6174c0-ae61-44bf-9eb0-97a72d05c434 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:13.397396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6174c0-ae61-44bf-9eb0-97a72d05c434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:13.410669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:13.437936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d32f79-1213-4fdf-b828-d1399d432ea5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:13.445307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:13.472921 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6174c0-ae61-44bf-9eb0-97a72d05c434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:13.473110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635af47e-2768-4b70-bb91-6701a12a24e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:13.473537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635af47e-2768-4b70-bb91-6701a12a24e3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:13.473854 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635af47e-2768-4b70-bb91-6701a12a24e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:13.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635af47e-2768-4b70-bb91-6701a12a24e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:13.474433 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6174c0-ae61-44bf-9eb0-97a72d05c434 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:13.474819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635af47e-2768-4b70-bb91-6701a12a24e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:13.475129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635af47e-2768-4b70-bb91-6701a12a24e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:13.476579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635af47e-2768-4b70-bb91-6701a12a24e3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:13.501119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 263/528] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:13.501119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635af47e-2768-4b70-bb91-6701a12a24e3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:13.586820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 266/529] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:13.587977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6174c0-ae61-44bf-9eb0-97a72d05c434 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:13.616640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635af47e-2768-4b70-bb91-6701a12a24e3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:13.633123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:13.654127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807a2a3-1000-4e24-924c-6285f351bfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:13.654725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807a2a3-1000-4e24-924c-6285f351bfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:13.655008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807a2a3-1000-4e24-924c-6285f351bfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:13.655284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807a2a3-1000-4e24-924c-6285f351bfb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:13.656007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807a2a3-1000-4e24-924c-6285f351bfb9 None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=9901608b62344115aa561d00bfe88c1c) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:13.656318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807a2a3-1000-4e24-924c-6285f351bfb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:13.657761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807a2a3-1000-4e24-924c-6285f351bfb9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:13.664209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 264/530] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:13 2022] GET /identity/v3/role_assignments?user.id=9901608b62344115aa561d00bfe88c1c&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:13.665035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807a2a3-1000-4e24-924c-6285f351bfb9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:13.699522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6174c0-ae61-44bf-9eb0-97a72d05c434 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:13.719608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:13.753777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc79c7e-4336-4f79-bc34-27579039ae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:13.754483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc79c7e-4336-4f79-bc34-27579039ae16 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:13.754768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc79c7e-4336-4f79-bc34-27579039ae16 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:13.755046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc79c7e-4336-4f79-bc34-27579039ae16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:13.771203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc79c7e-4336-4f79-bc34-27579039ae16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:13.772712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc79c7e-4336-4f79-bc34-27579039ae16 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-6bc79c7e-4336-4f79-bc34-27579039ae16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:13.796976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:13.798020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:13.799316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:13.799316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:13.799316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:13.802779 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 267/531] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:13.803916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc79c7e-4336-4f79-bc34-27579039ae16 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:13.827631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807a2a3-1000-4e24-924c-6285f351bfb9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:13.871331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4257483-1c1d-40ca-b60a-e03d65c62917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:13.871942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4257483-1c1d-40ca-b60a-e03d65c62917 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:13.872225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4257483-1c1d-40ca-b60a-e03d65c62917 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:13.872499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4257483-1c1d-40ca-b60a-e03d65c62917 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:13.873279 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4257483-1c1d-40ca-b60a-e03d65c62917 None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=ab54ee52004746eaadb5c2970e59afc3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:13.873595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4257483-1c1d-40ca-b60a-e03d65c62917 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:13.875214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4257483-1c1d-40ca-b60a-e03d65c62917 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:13.883353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 265/532] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:13 2022] GET /identity/v3/role_assignments?user.id=ab54ee52004746eaadb5c2970e59afc3&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:13.883353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4257483-1c1d-40ca-b60a-e03d65c62917 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:13.974986 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:13.987348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc79c7e-4336-4f79-bc34-27579039ae16 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:14.060033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:14.060553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face3fc8-8240-4d27-8a1c-c761bedd9895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:14.060969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face3fc8-8240-4d27-8a1c-c761bedd9895 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:14.061253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face3fc8-8240-4d27-8a1c-c761bedd9895 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:14.061526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face3fc8-8240-4d27-8a1c-c761bedd9895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:14.077906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4257483-1c1d-40ca-b60a-e03d65c62917 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:14.082264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face3fc8-8240-4d27-8a1c-c761bedd9895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:14.083750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face3fc8-8240-4d27-8a1c-c761bedd9895 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-face3fc8-8240-4d27-8a1c-c761bedd9895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:14.105999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:14.106997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:14.107867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:14.107867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:14.107867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:14.107867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 268/533] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:14.115770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face3fc8-8240-4d27-8a1c-c761bedd9895 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:14.131974 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b26732-7492-4d4d-93cf-d218e7eead63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:14.132929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b26732-7492-4d4d-93cf-d218e7eead63 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:14.133417 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b26732-7492-4d4d-93cf-d218e7eead63 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:14.133976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b26732-7492-4d4d-93cf-d218e7eead63 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:14.134951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b26732-7492-4d4d-93cf-d218e7eead63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:14.135497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b26732-7492-4d4d-93cf-d218e7eead63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:14.137078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b26732-7492-4d4d-93cf-d218e7eead63 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:14.150879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 266/534] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:14.159108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b26732-7492-4d4d-93cf-d218e7eead63 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:14.272179 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face3fc8-8240-4d27-8a1c-c761bedd9895 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:14.315831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bb32f-15cb-4d4d-b21d-d97b289e970d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:14.316443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bb32f-15cb-4d4d-b21d-d97b289e970d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:14.316721 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bb32f-15cb-4d4d-b21d-d97b289e970d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:14.316992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bb32f-15cb-4d4d-b21d-d97b289e970d None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:14.318058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bb32f-15cb-4d4d-b21d-d97b289e970d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:14.318373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bb32f-15cb-4d4d-b21d-d97b289e970d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:14.319746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bb32f-15cb-4d4d-b21d-d97b289e970d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:14.334946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 269/535] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:14.341068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bb32f-15cb-4d4d-b21d-d97b289e970d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:14.350643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:14.353239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b26732-7492-4d4d-93cf-d218e7eead63 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:14.430185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71a9fd-6252-42d4-841f-155d0c4ea837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:14.430822 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71a9fd-6252-42d4-841f-155d0c4ea837 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:14.431101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71a9fd-6252-42d4-841f-155d0c4ea837 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:14.431375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71a9fd-6252-42d4-841f-155d0c4ea837 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:14.459109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71a9fd-6252-42d4-841f-155d0c4ea837 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:14.460579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71a9fd-6252-42d4-841f-155d0c4ea837 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4d71a9fd-6252-42d4-841f-155d0c4ea837 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:14.474133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:14.477662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:14.479167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:14.479167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:14.479167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:14.479167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:14.479167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:14.479167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:14.479167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:14.479167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:14.479167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 267/536] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:14.486349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71a9fd-6252-42d4-841f-155d0c4ea837 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:14.518040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:14.553164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bb32f-15cb-4d4d-b21d-d97b289e970d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:14.628729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8bfb1-859c-47fe-b9d5-5c7ea01b80a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:14.629356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8bfb1-859c-47fe-b9d5-5c7ea01b80a0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:14.629636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8bfb1-859c-47fe-b9d5-5c7ea01b80a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:14.629930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8bfb1-859c-47fe-b9d5-5c7ea01b80a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:14.644828 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8bfb1-859c-47fe-b9d5-5c7ea01b80a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:14.646434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8bfb1-859c-47fe-b9d5-5c7ea01b80a0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:14.654868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71a9fd-6252-42d4-841f-155d0c4ea837 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-26f8bfb1-859c-47fe-b9d5-5c7ea01b80a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:14.664885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:14.665815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:14.667374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:14.667374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:14.667374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:14.667374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:14.667374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:14.667374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:14.667374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:14.667374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:14.671677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 270/537] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:14.672755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8bfb1-859c-47fe-b9d5-5c7ea01b80a0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:14.703334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a5b4a-2093-4c48-bdc4-cf31c61904b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:14.704323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a5b4a-2093-4c48-bdc4-cf31c61904b3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:14.704776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a5b4a-2093-4c48-bdc4-cf31c61904b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:14.705275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a5b4a-2093-4c48-bdc4-cf31c61904b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:14.706233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a5b4a-2093-4c48-bdc4-cf31c61904b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:14.706708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a5b4a-2093-4c48-bdc4-cf31c61904b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:14.708369 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a5b4a-2093-4c48-bdc4-cf31c61904b3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:14.722363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 268/538] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:14.730707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a5b4a-2093-4c48-bdc4-cf31c61904b3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:14.760585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:14.798768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8bfb1-859c-47fe-b9d5-5c7ea01b80a0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:14.837796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d228fb-6af7-4bc4-9ee3-1f4020fbf781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:14.840026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d228fb-6af7-4bc4-9ee3-1f4020fbf781 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:14.840302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d228fb-6af7-4bc4-9ee3-1f4020fbf781 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:14.840573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d228fb-6af7-4bc4-9ee3-1f4020fbf781 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:14.841266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d228fb-6af7-4bc4-9ee3-1f4020fbf781 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:14.841570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d228fb-6af7-4bc4-9ee3-1f4020fbf781 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:14.843033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d228fb-6af7-4bc4-9ee3-1f4020fbf781 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:14.869781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:14.871543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 271/539] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:14.877591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d228fb-6af7-4bc4-9ee3-1f4020fbf781 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:14.921109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a5b4a-2093-4c48-bdc4-cf31c61904b3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:15.003853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5793e2-9c5c-4869-abbd-9a85fb70786f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:15.004816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5793e2-9c5c-4869-abbd-9a85fb70786f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:15.005291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5793e2-9c5c-4869-abbd-9a85fb70786f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:15.005781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5793e2-9c5c-4869-abbd-9a85fb70786f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:15.006720 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5793e2-9c5c-4869-abbd-9a85fb70786f None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=727f7207d82640a3a228dfb83814148b) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:15.007239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5793e2-9c5c-4869-abbd-9a85fb70786f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:15.008846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5793e2-9c5c-4869-abbd-9a85fb70786f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:15.017325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 269/540] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:14 2022] GET /identity/v3/role_assignments?user.id=727f7207d82640a3a228dfb83814148b&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:15.052022 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d228fb-6af7-4bc4-9ee3-1f4020fbf781 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:15.100448 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:15.116174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a3978-5a2c-411e-bd7a-a7bbf15efbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:15.117148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a3978-5a2c-411e-bd7a-a7bbf15efbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:15.117618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a3978-5a2c-411e-bd7a-a7bbf15efbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:15.118102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a3978-5a2c-411e-bd7a-a7bbf15efbe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:15.119046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a3978-5a2c-411e-bd7a-a7bbf15efbe7 None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=6336f0c469b84db78a90cf8435c7dbe4) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:15.119790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a3978-5a2c-411e-bd7a-a7bbf15efbe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:15.121535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a3978-5a2c-411e-bd7a-a7bbf15efbe7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:15.129688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 272/541] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:14 2022] GET /identity/v3/role_assignments?user.id=6336f0c469b84db78a90cf8435c7dbe4&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:15.139327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec0c20-0268-4840-8cda-46669e91871f None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:15.140310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec0c20-0268-4840-8cda-46669e91871f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:15.140783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec0c20-0268-4840-8cda-46669e91871f None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:15.142747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 270/542] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:15.148897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a405e6-1695-4736-96fd-557a06f5ae33 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:15.149202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a405e6-1695-4736-96fd-557a06f5ae33 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:15.149457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a405e6-1695-4736-96fd-557a06f5ae33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:15.175730 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-59a405e6-1695-4736-96fd-557a06f5ae33 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:15.176944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-59a405e6-1695-4736-96fd-557a06f5ae33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:15.276669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 273/543] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:15.612410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec0c20-0268-4840-8cda-46669e91871f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:15.738533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec0c20-0268-4840-8cda-46669e91871f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:15.785671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48426a9e-6f1f-4da2-96c1-b2a5c02a3d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:15.786746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48426a9e-6f1f-4da2-96c1-b2a5c02a3d5e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:15.787247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48426a9e-6f1f-4da2-96c1-b2a5c02a3d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:15.787719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48426a9e-6f1f-4da2-96c1-b2a5c02a3d5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:15.810905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:15.817815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48426a9e-6f1f-4da2-96c1-b2a5c02a3d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-48426a9e-6f1f-4da2-96c1-b2a5c02a3d5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:15.824243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:15.825160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:15.826025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:15.834413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a405e6-1695-4736-96fd-557a06f5ae33 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:15.840916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 271/544] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:15.976314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a405e6-1695-4736-96fd-557a06f5ae33 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:16.003710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:16.024144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c56fd-086d-4ed0-9c71-3a5dd718cc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:16.024758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c56fd-086d-4ed0-9c71-3a5dd718cc4e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:16.025033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c56fd-086d-4ed0-9c71-3a5dd718cc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:16.025437 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c56fd-086d-4ed0-9c71-3a5dd718cc4e None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:16.026451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c56fd-086d-4ed0-9c71-3a5dd718cc4e None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:16.026810 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c56fd-086d-4ed0-9c71-3a5dd718cc4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:16.028184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c56fd-086d-4ed0-9c71-3a5dd718cc4e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:16.043236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48426a9e-6f1f-4da2-96c1-b2a5c02a3d5e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:16.044285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 274/545] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:16.184935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48426a9e-6f1f-4da2-96c1-b2a5c02a3d5e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:16.232254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f17a09-e5eb-40a6-8708-5d4958cb33ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:16.233174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f17a09-e5eb-40a6-8708-5d4958cb33ab None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:16.233636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f17a09-e5eb-40a6-8708-5d4958cb33ab None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:16.234129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f17a09-e5eb-40a6-8708-5d4958cb33ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:16.245112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f17a09-e5eb-40a6-8708-5d4958cb33ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:16.246820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f17a09-e5eb-40a6-8708-5d4958cb33ab None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c5f17a09-e5eb-40a6-8708-5d4958cb33ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:16.264318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:16.265348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:16.270435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:16.270435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:16.270435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:16.272170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 272/546] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:16.282845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c56fd-086d-4ed0-9c71-3a5dd718cc4e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:16.308448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82800b27-8448-46b6-a777-bd08c1cd27a5 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:16.309488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82800b27-8448-46b6-a777-bd08c1cd27a5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:16.310186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82800b27-8448-46b6-a777-bd08c1cd27a5 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:16.311991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 273/547] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:16.317003 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd67034-4612-44c7-be4f-2c944fcbe495 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:16.317542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd67034-4612-44c7-be4f-2c944fcbe495 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:16.318196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd67034-4612-44c7-be4f-2c944fcbe495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:16.349040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1dd67034-4612-44c7-be4f-2c944fcbe495 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:16.351413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1dd67034-4612-44c7-be4f-2c944fcbe495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:16.410140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:16.436407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 274/548] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:30:16.492416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c56fd-086d-4ed0-9c71-3a5dd718cc4e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:16.548732 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0667b84-5fcb-4979-a5c2-43a97b9c9c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:16.549336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667b84-5fcb-4979-a5c2-43a97b9c9c89 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:16.549609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667b84-5fcb-4979-a5c2-43a97b9c9c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:16.549904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667b84-5fcb-4979-a5c2-43a97b9c9c89 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:16.550544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0667b84-5fcb-4979-a5c2-43a97b9c9c89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:16.550841 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0667b84-5fcb-4979-a5c2-43a97b9c9c89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:16.552200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0667b84-5fcb-4979-a5c2-43a97b9c9c89 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:16.564790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 275/549] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:16.570108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd67034-4612-44c7-be4f-2c944fcbe495 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:16.610653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180c979-2c5f-4aa2-a03f-603bee50d585 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:16.610927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180c979-2c5f-4aa2-a03f-603bee50d585 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:16.611196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180c979-2c5f-4aa2-a03f-603bee50d585 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:16.612526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 276/550] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:30:16.619521 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2ab6-369d-4ed7-8962-2b183fa99629 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:16.619814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2ab6-369d-4ed7-8962-2b183fa99629 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:16.620099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2ab6-369d-4ed7-8962-2b183fa99629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:16.650523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-5abe2ab6-369d-4ed7-8962-2b183fa99629 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:16.653586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5abe2ab6-369d-4ed7-8962-2b183fa99629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:16.722389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:16.743214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd67034-4612-44c7-be4f-2c944fcbe495 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:16.781800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 277/551] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:16.809669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9f3cc-c2bd-4550-adbf-9ae179ab0cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:16.810347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9f3cc-c2bd-4550-adbf-9ae179ab0cac None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:16.810629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9f3cc-c2bd-4550-adbf-9ae179ab0cac None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:16.810902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9f3cc-c2bd-4550-adbf-9ae179ab0cac None admin] PATH_INFO: `/v3/users/swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:16.821488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9f3cc-c2bd-4550-adbf-9ae179ab0cac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:16.823075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9f3cc-c2bd-4550-adbf-9ae179ab0cac None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-fdd9f3cc-c2bd-4550-adbf-9ae179ab0cac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:16.842254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:16.843217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:16.844215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:16.844215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:16.844215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:16.844215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:16.844215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:16.844215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:16.844215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:16.844215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:16.844215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:16.844215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:16.844832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 275/552] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:16.848878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe2ab6-369d-4ed7-8962-2b183fa99629 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:16.873748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9f3cc-c2bd-4550-adbf-9ae179ab0cac None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:16.938110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:17.027507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe2ab6-369d-4ed7-8962-2b183fa99629 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.043489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9f3cc-c2bd-4550-adbf-9ae179ab0cac None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.081408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a4230-874b-4666-aa11-d72414020ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:17.082478 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a4230-874b-4666-aa11-d72414020ffd None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.082957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a4230-874b-4666-aa11-d72414020ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.083447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a4230-874b-4666-aa11-d72414020ffd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.097797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a4230-874b-4666-aa11-d72414020ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ee1a4230-874b-4666-aa11-d72414020ffd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:17.105946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:17.106892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:17.107747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:17.111541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 276/553] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:17.112581 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a4230-874b-4666-aa11-d72414020ffd None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:17.114755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cf925-b956-4545-85bf-ac47b515b899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:17.115547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cf925-b956-4545-85bf-ac47b515b899 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.116226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cf925-b956-4545-85bf-ac47b515b899 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.116644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cf925-b956-4545-85bf-ac47b515b899 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.117528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cf925-b956-4545-85bf-ac47b515b899 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:17.117864 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cf925-b956-4545-85bf-ac47b515b899 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:17.119914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cf925-b956-4545-85bf-ac47b515b899 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:17.138428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 278/554] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Sep 17 03:30:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:17.143802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cf925-b956-4545-85bf-ac47b515b899 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:17.178239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:17.221380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:17.225428 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a4230-874b-4666-aa11-d72414020ffd None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.256955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cf925-b956-4545-85bf-ac47b515b899 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.262597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3c2e9-78dc-4b3f-90a2-28c3cb690631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:17.263410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3c2e9-78dc-4b3f-90a2-28c3cb690631 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.263839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3c2e9-78dc-4b3f-90a2-28c3cb690631 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.264285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3c2e9-78dc-4b3f-90a2-28c3cb690631 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.275066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3c2e9-78dc-4b3f-90a2-28c3cb690631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-25e3c2e9-78dc-4b3f-90a2-28c3cb690631 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:17.281275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:17.282270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:17.283098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:17.285641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 277/555] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:17.286860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3c2e9-78dc-4b3f-90a2-28c3cb690631 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:17.295726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7a327-da85-412c-b579-63fd525a8cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:17.296421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7a327-da85-412c-b579-63fd525a8cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.296701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7a327-da85-412c-b579-63fd525a8cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.296970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7a327-da85-412c-b579-63fd525a8cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.297635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7a327-da85-412c-b579-63fd525a8cb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:17.297982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7a327-da85-412c-b579-63fd525a8cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:17.299425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7a327-da85-412c-b579-63fd525a8cb0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:17.310955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 279/556] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:17.311822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7a327-da85-412c-b579-63fd525a8cb0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:17.400513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3c2e9-78dc-4b3f-90a2-28c3cb690631 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.411488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:17.422032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7a327-da85-412c-b579-63fd525a8cb0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.445100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4028b14-8494-42d3-a7fd-1a9abdf74066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:17.445717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4028b14-8494-42d3-a7fd-1a9abdf74066 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.446031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4028b14-8494-42d3-a7fd-1a9abdf74066 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.446312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4028b14-8494-42d3-a7fd-1a9abdf74066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.466405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceacf1-df7d-4f1d-a898-ab71ac4daf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:17.467022 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceacf1-df7d-4f1d-a898-ab71ac4daf63 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.467302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceacf1-df7d-4f1d-a898-ab71ac4daf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.467574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceacf1-df7d-4f1d-a898-ab71ac4daf63 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.468652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdceacf1-df7d-4f1d-a898-ab71ac4daf63 None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:17.468962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdceacf1-df7d-4f1d-a898-ab71ac4daf63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:17.470369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdceacf1-df7d-4f1d-a898-ab71ac4daf63 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:17.479936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:17.488069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 280/557] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:17.496096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceacf1-df7d-4f1d-a898-ab71ac4daf63 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:17.529381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4028b14-8494-42d3-a7fd-1a9abdf74066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:17.531242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4028b14-8494-42d3-a7fd-1a9abdf74066 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:17.601634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceacf1-df7d-4f1d-a898-ab71ac4daf63 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.643753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60295fe-be23-47d4-acce-21a4b22181cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:17.644468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60295fe-be23-47d4-acce-21a4b22181cb None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.644753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60295fe-be23-47d4-acce-21a4b22181cb None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.645026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60295fe-be23-47d4-acce-21a4b22181cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.648586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 278/558] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:17.649715 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4028b14-8494-42d3-a7fd-1a9abdf74066 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:17.656543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60295fe-be23-47d4-acce-21a4b22181cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:17.658030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60295fe-be23-47d4-acce-21a4b22181cb None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-f60295fe-be23-47d4-acce-21a4b22181cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:17.674386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:17.675390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:17.676289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:17.676289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:17.676289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:17.680250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 281/559] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:17.681224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60295fe-be23-47d4-acce-21a4b22181cb None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:17.691461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:17.718990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:17.766157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4028b14-8494-42d3-a7fd-1a9abdf74066 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.796527 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60295fe-be23-47d4-acce-21a4b22181cb None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.812765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9df35-ae15-4d6c-a56a-bb2b8f359bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:17.813224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9df35-ae15-4d6c-a56a-bb2b8f359bfe None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.813511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9df35-ae15-4d6c-a56a-bb2b8f359bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.813821 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9df35-ae15-4d6c-a56a-bb2b8f359bfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.825168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9df35-ae15-4d6c-a56a-bb2b8f359bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:17.826665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9df35-ae15-4d6c-a56a-bb2b8f359bfe None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:17.837465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfdc6d-c647-4609-8b15-9e0cb2994a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:17.838822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfdc6d-c647-4609-8b15-9e0cb2994a26 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.839302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfdc6d-c647-4609-8b15-9e0cb2994a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.839722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfdc6d-c647-4609-8b15-9e0cb2994a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-46d9df35-ae15-4d6c-a56a-bb2b8f359bfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:17.841365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:17.842506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:17.843406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:17.843406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:17.843406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:17.843406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 279/560] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:17.843663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9df35-ae15-4d6c-a56a-bb2b8f359bfe None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:17.857462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfdc6d-c647-4609-8b15-9e0cb2994a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:17.858886 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfdc6d-c647-4609-8b15-9e0cb2994a26 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-10cfdc6d-c647-4609-8b15-9e0cb2994a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:17.872906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:17.873893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:17.874881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:17.874881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:17.874881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:17.874881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 282/561] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:17.875268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1ea19-25ea-4ade-a9e4-1cf90cee06ce None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.875515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1ea19-25ea-4ade-a9e4-1cf90cee06ce None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.875791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1ea19-25ea-4ade-a9e4-1cf90cee06ce None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.877012 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 283/562] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:17.877697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1ea19-25ea-4ade-a9e4-1cf90cee06ce None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:17.937271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:17.950810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9df35-ae15-4d6c-a56a-bb2b8f359bfe None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.963767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:17.979984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1ea19-25ea-4ade-a9e4-1cf90cee06ce None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:17.984045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb541c9-b52d-4521-9234-8cbc082f2f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:17.984640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb541c9-b52d-4521-9234-8cbc082f2f77 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:17.984919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb541c9-b52d-4521-9234-8cbc082f2f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:17.985195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb541c9-b52d-4521-9234-8cbc082f2f77 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:17.985857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb541c9-b52d-4521-9234-8cbc082f2f77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:17.986164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb541c9-b52d-4521-9234-8cbc082f2f77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:17.987534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb541c9-b52d-4521-9234-8cbc082f2f77 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:17.999878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 280/563] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:18.001063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58130745-e3da-452c-8870-73b84207ad24 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.001355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58130745-e3da-452c-8870-73b84207ad24 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.001632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58130745-e3da-452c-8870-73b84207ad24 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.002876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 281/564] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:18.003571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58130745-e3da-452c-8870-73b84207ad24 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.018323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7678d5-5330-4a2a-af35-8f2de6cd554c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.018930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7678d5-5330-4a2a-af35-8f2de6cd554c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.019210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7678d5-5330-4a2a-af35-8f2de6cd554c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.019487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7678d5-5330-4a2a-af35-8f2de6cd554c None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.020126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7678d5-5330-4a2a-af35-8f2de6cd554c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:18.020455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7678d5-5330-4a2a-af35-8f2de6cd554c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:18.021854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7678d5-5330-4a2a-af35-8f2de6cd554c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:18.034253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 284/565] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:18.035398 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f2f7f-43e1-4c8e-963b-d2dbc1dc209d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.035682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f2f7f-43e1-4c8e-963b-d2dbc1dc209d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.035957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f2f7f-43e1-4c8e-963b-d2dbc1dc209d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.053098 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d25f2f7f-43e1-4c8e-963b-d2dbc1dc209d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:18.054318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d25f2f7f-43e1-4c8e-963b-d2dbc1dc209d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:18.110027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58130745-e3da-452c-8870-73b84207ad24 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:18.123785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 285/566] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:18.124628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f2f7f-43e1-4c8e-963b-d2dbc1dc209d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.152550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6194d2-4999-4c96-86ce-fa3b34096ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.153264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6194d2-4999-4c96-86ce-fa3b34096ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.153540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6194d2-4999-4c96-86ce-fa3b34096ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.153953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6194d2-4999-4c96-86ce-fa3b34096ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.154586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6194d2-4999-4c96-86ce-fa3b34096ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:18.154883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6194d2-4999-4c96-86ce-fa3b34096ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:18.156231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6194d2-4999-4c96-86ce-fa3b34096ca1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:18.174154 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 282/567] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:18.175173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ad6cf-39e1-4575-a8a0-16d3a42c14a9 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.175462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ad6cf-39e1-4575-a8a0-16d3a42c14a9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.175739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ad6cf-39e1-4575-a8a0-16d3a42c14a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.179504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:18.194340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-cf3ad6cf-39e1-4575-a8a0-16d3a42c14a9 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:18.195540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cf3ad6cf-39e1-4575-a8a0-16d3a42c14a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:18.222410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:18.236531 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f2f7f-43e1-4c8e-963b-d2dbc1dc209d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:18.267099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 283/568] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:18.267844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ad6cf-39e1-4575-a8a0-16d3a42c14a9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.283682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00d3d8-6a9e-4198-8e23-98db051816d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.284855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00d3d8-6a9e-4198-8e23-98db051816d3 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.285326 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00d3d8-6a9e-4198-8e23-98db051816d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.286031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00d3d8-6a9e-4198-8e23-98db051816d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.294765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00d3d8-6a9e-4198-8e23-98db051816d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:18.296277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00d3d8-6a9e-4198-8e23-98db051816d3 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-cd00d3d8-6a9e-4198-8e23-98db051816d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:18.308817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:18.309843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:18.310834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:18.310834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:18.310834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:18.310834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:18.310834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:18.310834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:18.310834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:18.310834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:18.310834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:18.310834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:18.314809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 286/569] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:18.315711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00d3d8-6a9e-4198-8e23-98db051816d3 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.391542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ad6cf-39e1-4575-a8a0-16d3a42c14a9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:18.430735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7461a9-b5a0-40e1-84bf-a6b7d4073a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.431700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7461a9-b5a0-40e1-84bf-a6b7d4073a37 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.432439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7461a9-b5a0-40e1-84bf-a6b7d4073a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.432725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7461a9-b5a0-40e1-84bf-a6b7d4073a37 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.438822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00d3d8-6a9e-4198-8e23-98db051816d3 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:18.440457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7461a9-b5a0-40e1-84bf-a6b7d4073a37 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:18.441943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7461a9-b5a0-40e1-84bf-a6b7d4073a37 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:18.449099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-5e7461a9-b5a0-40e1-84bf-a6b7d4073a37 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:18.455028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:18.456036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 284/570] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:18.457020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7461a9-b5a0-40e1-84bf-a6b7d4073a37 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.489993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:18.495154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0538a7-298d-468a-ae81-d016fd6a061f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.495536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0538a7-298d-468a-ae81-d016fd6a061f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.495812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0538a7-298d-468a-ae81-d016fd6a061f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.496079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0538a7-298d-468a-ae81-d016fd6a061f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.503140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0538a7-298d-468a-ae81-d016fd6a061f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:18.504585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0538a7-298d-468a-ae81-d016fd6a061f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-6a0538a7-298d-468a-ae81-d016fd6a061f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:18.516912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:18.517875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:18.518769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:18.518769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:18.518769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:18.518769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:18.518769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:18.518769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:18.518769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:18.518769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:18.518769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 287/571] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:18.518769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0538a7-298d-468a-ae81-d016fd6a061f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.574993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7461a9-b5a0-40e1-84bf-a6b7d4073a37 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:18.610551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46854ac-eaf1-4e56-955b-925769a4d0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.611152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46854ac-eaf1-4e56-955b-925769a4d0fb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.611433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46854ac-eaf1-4e56-955b-925769a4d0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.611710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46854ac-eaf1-4e56-955b-925769a4d0fb None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.612434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46854ac-eaf1-4e56-955b-925769a4d0fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:18.612739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46854ac-eaf1-4e56-955b-925769a4d0fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:18.614195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46854ac-eaf1-4e56-955b-925769a4d0fb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:18.624049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0538a7-298d-468a-ae81-d016fd6a061f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:18.631779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 285/572] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:18.632593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46854ac-eaf1-4e56-955b-925769a4d0fb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.658073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc502ba8-7547-4a33-9659-716952b1d875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.658776 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc502ba8-7547-4a33-9659-716952b1d875 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.659126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc502ba8-7547-4a33-9659-716952b1d875 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.659463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc502ba8-7547-4a33-9659-716952b1d875 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.670025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc502ba8-7547-4a33-9659-716952b1d875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-cc502ba8-7547-4a33-9659-716952b1d875 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:18.675152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:18.676243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:18.677275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 288/573] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:18.678119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc502ba8-7547-4a33-9659-716952b1d875 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.705645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:18.717532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:18.735493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46854ac-eaf1-4e56-955b-925769a4d0fb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:18.775201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b733b9-4641-40e2-8d3b-b9314b30a7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.775825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b733b9-4641-40e2-8d3b-b9314b30a7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.776110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b733b9-4641-40e2-8d3b-b9314b30a7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.776386 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b733b9-4641-40e2-8d3b-b9314b30a7b1 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.777117 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b733b9-4641-40e2-8d3b-b9314b30a7b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:18.777422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b733b9-4641-40e2-8d3b-b9314b30a7b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:18.778884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b733b9-4641-40e2-8d3b-b9314b30a7b1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:18.779590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc502ba8-7547-4a33-9659-716952b1d875 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:18.799159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 286/574] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:18.800010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b733b9-4641-40e2-8d3b-b9314b30a7b1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.816496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc60334-e3ae-41d4-8f85-69e036ae5634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.817097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc60334-e3ae-41d4-8f85-69e036ae5634 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.817375 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc60334-e3ae-41d4-8f85-69e036ae5634 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.817709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc60334-e3ae-41d4-8f85-69e036ae5634 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.818436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc60334-e3ae-41d4-8f85-69e036ae5634 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:18.818742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc60334-e3ae-41d4-8f85-69e036ae5634 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:18.820180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc60334-e3ae-41d4-8f85-69e036ae5634 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:18.832750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 289/575] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:18.833612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc60334-e3ae-41d4-8f85-69e036ae5634 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.899457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b733b9-4641-40e2-8d3b-b9314b30a7b1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:18.921514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:18.934148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc31bd-aa6f-4481-8c4c-432e422ad710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.934305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc60334-e3ae-41d4-8f85-69e036ae5634 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:18.935114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc31bd-aa6f-4481-8c4c-432e422ad710 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.935114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc31bd-aa6f-4481-8c4c-432e422ad710 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.935281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc31bd-aa6f-4481-8c4c-432e422ad710 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.942114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:18.946075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc31bd-aa6f-4481-8c4c-432e422ad710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f9fc31bd-aa6f-4481-8c4c-432e422ad710 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:18.951150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:18.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 287/576] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:18.953421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc31bd-aa6f-4481-8c4c-432e422ad710 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:18.969418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b36b34-2cc2-4ddc-84cd-9afb849a000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:18.970043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b36b34-2cc2-4ddc-84cd-9afb849a000d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:18.970323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b36b34-2cc2-4ddc-84cd-9afb849a000d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:18.970595 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b36b34-2cc2-4ddc-84cd-9afb849a000d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:18.981019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b36b34-2cc2-4ddc-84cd-9afb849a000d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:18.982491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b36b34-2cc2-4ddc-84cd-9afb849a000d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-06b36b34-2cc2-4ddc-84cd-9afb849a000d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:18.997373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:18.998547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:18.999464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:18.999464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:18.999464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:18.999464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 290/577] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:18.999464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b36b34-2cc2-4ddc-84cd-9afb849a000d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.053122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc31bd-aa6f-4481-8c4c-432e422ad710 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.086883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbe651-e62f-4e38-89b0-2522ef0bf9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.087478 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe651-e62f-4e38-89b0-2522ef0bf9fe None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.088450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe651-e62f-4e38-89b0-2522ef0bf9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.088897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe651-e62f-4e38-89b0-2522ef0bf9fe None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.089304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe651-e62f-4e38-89b0-2522ef0bf9fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:19.089611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe651-e62f-4e38-89b0-2522ef0bf9fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.091007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe651-e62f-4e38-89b0-2522ef0bf9fe None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.098777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 288/578] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:19.099123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b36b34-2cc2-4ddc-84cd-9afb849a000d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.099551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbe651-e62f-4e38-89b0-2522ef0bf9fe None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.133161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ba9e-98fe-47c2-bb0e-4fbf3fadc0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.133809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ba9e-98fe-47c2-bb0e-4fbf3fadc0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.134085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ba9e-98fe-47c2-bb0e-4fbf3fadc0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.134352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ba9e-98fe-47c2-bb0e-4fbf3fadc0c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.144475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:19.146865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58ba9e-98fe-47c2-bb0e-4fbf3fadc0c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.148300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58ba9e-98fe-47c2-bb0e-4fbf3fadc0c9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.157756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-1a58ba9e-98fe-47c2-bb0e-4fbf3fadc0c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:19.162500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:19.163605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:19.164507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:19.164507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:19.164507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:19.164507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 291/579] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:19.164507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ba9e-98fe-47c2-bb0e-4fbf3fadc0c9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.203083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbe651-e62f-4e38-89b0-2522ef0bf9fe None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.237000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad04686-2076-4cc5-8031-3b109d2e4453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.237636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad04686-2076-4cc5-8031-3b109d2e4453 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.237891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad04686-2076-4cc5-8031-3b109d2e4453 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.238169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad04686-2076-4cc5-8031-3b109d2e4453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.238896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad04686-2076-4cc5-8031-3b109d2e4453 None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=c1136770f79a47f6a2af0b0a32259209) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:19.239204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad04686-2076-4cc5-8031-3b109d2e4453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.240639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad04686-2076-4cc5-8031-3b109d2e4453 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.246928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 289/580] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/role_assignments?user.id=c1136770f79a47f6a2af0b0a32259209&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:19.247715 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad04686-2076-4cc5-8031-3b109d2e4453 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.269318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ba9e-98fe-47c2-bb0e-4fbf3fadc0c9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.306336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672d9fc-d878-4abc-a436-e37b509c2d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.307562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672d9fc-d878-4abc-a436-e37b509c2d5a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.307781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672d9fc-d878-4abc-a436-e37b509c2d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.308176 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672d9fc-d878-4abc-a436-e37b509c2d5a None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.308882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672d9fc-d878-4abc-a436-e37b509c2d5a None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:19.309168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672d9fc-d878-4abc-a436-e37b509c2d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.310738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672d9fc-d878-4abc-a436-e37b509c2d5a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.319795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 292/581] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:19.320664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672d9fc-d878-4abc-a436-e37b509c2d5a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.381225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad04686-2076-4cc5-8031-3b109d2e4453 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.394867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:19.408600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:19.419892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38d725-32f2-4b90-80f1-5be72747c6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.420658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38d725-32f2-4b90-80f1-5be72747c6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.421041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38d725-32f2-4b90-80f1-5be72747c6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.421442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38d725-32f2-4b90-80f1-5be72747c6f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.422233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38d725-32f2-4b90-80f1-5be72747c6f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:19.423228 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38d725-32f2-4b90-80f1-5be72747c6f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.424165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38d725-32f2-4b90-80f1-5be72747c6f6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.437634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 290/582] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:19.446881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38d725-32f2-4b90-80f1-5be72747c6f6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.451453 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672d9fc-d878-4abc-a436-e37b509c2d5a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.487515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa48c5-e756-495d-aa24-203bfe3a2def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.488066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa48c5-e756-495d-aa24-203bfe3a2def None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.488345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa48c5-e756-495d-aa24-203bfe3a2def None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.488621 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa48c5-e756-495d-aa24-203bfe3a2def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.500247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa48c5-e756-495d-aa24-203bfe3a2def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.501700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa48c5-e756-495d-aa24-203bfe3a2def None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-2faa48c5-e756-495d-aa24-203bfe3a2def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:19.516512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:19.517500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:19.523194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:19.523194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:19.523194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:19.523194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 293/583] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:19.523194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa48c5-e756-495d-aa24-203bfe3a2def None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.550704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38d725-32f2-4b90-80f1-5be72747c6f6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.586332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d56219-4134-4e6c-98e1-c2d8e3cbba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.586938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d56219-4134-4e6c-98e1-c2d8e3cbba01 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.587216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d56219-4134-4e6c-98e1-c2d8e3cbba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.587487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d56219-4134-4e6c-98e1-c2d8e3cbba01 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.588827 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d56219-4134-4e6c-98e1-c2d8e3cbba01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:19.589117 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d56219-4134-4e6c-98e1-c2d8e3cbba01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.590496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d56219-4134-4e6c-98e1-c2d8e3cbba01 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.601957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 291/584] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:19.602767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d56219-4134-4e6c-98e1-c2d8e3cbba01 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.636344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa48c5-e756-495d-aa24-203bfe3a2def None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.644474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:19.649425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:19.672448 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1dbc9-b17c-4ea7-838c-3e1c2c3c44d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.673070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1dbc9-b17c-4ea7-838c-3e1c2c3c44d0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.673345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1dbc9-b17c-4ea7-838c-3e1c2c3c44d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.673613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1dbc9-b17c-4ea7-838c-3e1c2c3c44d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.684765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1dbc9-b17c-4ea7-838c-3e1c2c3c44d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.686254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1dbc9-b17c-4ea7-838c-3e1c2c3c44d0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-84c1dbc9-b17c-4ea7-838c-3e1c2c3c44d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:19.701579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:19.702706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:19.704019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:19.704019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:19.704019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:19.704019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 294/585] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:19.704019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1dbc9-b17c-4ea7-838c-3e1c2c3c44d0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.710226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d56219-4134-4e6c-98e1-c2d8e3cbba01 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.748690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021626a-9b27-41f5-bf19-b939778c2650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.749236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021626a-9b27-41f5-bf19-b939778c2650 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.749513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021626a-9b27-41f5-bf19-b939778c2650 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.749811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021626a-9b27-41f5-bf19-b939778c2650 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.756483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021626a-9b27-41f5-bf19-b939778c2650 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.757924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021626a-9b27-41f5-bf19-b939778c2650 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e021626a-9b27-41f5-bf19-b939778c2650 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:19.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:19.771042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:19.771967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:19.771967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:19.771967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:19.771967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:19.771967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:19.771967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:19.771967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:19.771967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:19.771967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 292/586] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:19.772498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021626a-9b27-41f5-bf19-b939778c2650 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.821970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1dbc9-b17c-4ea7-838c-3e1c2c3c44d0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.857324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f7139-875b-40cf-b0ce-5b8c9d641d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.857954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f7139-875b-40cf-b0ce-5b8c9d641d7c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.858229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f7139-875b-40cf-b0ce-5b8c9d641d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.858498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f7139-875b-40cf-b0ce-5b8c9d641d7c None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.859129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f7139-875b-40cf-b0ce-5b8c9d641d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:19.859429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f7139-875b-40cf-b0ce-5b8c9d641d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.860793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f7139-875b-40cf-b0ce-5b8c9d641d7c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.872427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 295/587] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:19.873209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f7139-875b-40cf-b0ce-5b8c9d641d7c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.887127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021626a-9b27-41f5-bf19-b939778c2650 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:19.890490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:19.894705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:19.923798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc353ae-2780-468e-9731-62d4a15a5e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:19.924429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc353ae-2780-468e-9731-62d4a15a5e60 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:19.924705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc353ae-2780-468e-9731-62d4a15a5e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:19.924979 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc353ae-2780-468e-9731-62d4a15a5e60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:19.931892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc353ae-2780-468e-9731-62d4a15a5e60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:19.933556 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc353ae-2780-468e-9731-62d4a15a5e60 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-5fc353ae-2780-468e-9731-62d4a15a5e60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:19.945405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:19.946524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:19.947705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:19.947705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:19.947705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:19.947705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:19.947705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:19.947705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:19.947705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:19.947705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:19.947705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 293/588] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:19.947705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc353ae-2780-468e-9731-62d4a15a5e60 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:19.990954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f7139-875b-40cf-b0ce-5b8c9d641d7c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:20.028273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845ddd6-13b0-4f44-8e8d-3fdc6a6dc22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:20.028800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845ddd6-13b0-4f44-8e8d-3fdc6a6dc22d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:20.029075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845ddd6-13b0-4f44-8e8d-3fdc6a6dc22d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:20.029343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845ddd6-13b0-4f44-8e8d-3fdc6a6dc22d None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:20.030001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845ddd6-13b0-4f44-8e8d-3fdc6a6dc22d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:20.030731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845ddd6-13b0-4f44-8e8d-3fdc6a6dc22d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:20.032449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845ddd6-13b0-4f44-8e8d-3fdc6a6dc22d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:20.048796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 296/589] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:20.048796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845ddd6-13b0-4f44-8e8d-3fdc6a6dc22d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:20.061338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc353ae-2780-468e-9731-62d4a15a5e60 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:20.097456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a4409-9459-4331-9e71-6e96c97908df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:20.098092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a4409-9459-4331-9e71-6e96c97908df None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:20.098376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a4409-9459-4331-9e71-6e96c97908df None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:20.098656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a4409-9459-4331-9e71-6e96c97908df None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:20.099350 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a4409-9459-4331-9e71-6e96c97908df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:20.099664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a4409-9459-4331-9e71-6e96c97908df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:20.101102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a4409-9459-4331-9e71-6e96c97908df None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:20.113522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 294/590] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:20.119568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a4409-9459-4331-9e71-6e96c97908df None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:20.136862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:20.146546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:20.164546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845ddd6-13b0-4f44-8e8d-3fdc6a6dc22d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:20.202948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beeffaf-41b6-491f-83be-f6c40d35b7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:20.203638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beeffaf-41b6-491f-83be-f6c40d35b7ff None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:20.203838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beeffaf-41b6-491f-83be-f6c40d35b7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:20.204107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beeffaf-41b6-491f-83be-f6c40d35b7ff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:20.212045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beeffaf-41b6-491f-83be-f6c40d35b7ff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:20.214050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beeffaf-41b6-491f-83be-f6c40d35b7ff None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:20.223946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a4409-9459-4331-9e71-6e96c97908df None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-7beeffaf-41b6-491f-83be-f6c40d35b7ff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:20.226780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:20.227750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 297/591] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:20.229223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beeffaf-41b6-491f-83be-f6c40d35b7ff None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:20.265124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e0502-10a2-4f03-8e2b-196568d1b6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:20.265840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e0502-10a2-4f03-8e2b-196568d1b6ed None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:20.266011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e0502-10a2-4f03-8e2b-196568d1b6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:20.266285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e0502-10a2-4f03-8e2b-196568d1b6ed None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:20.266992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e0502-10a2-4f03-8e2b-196568d1b6ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:20.267296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e0502-10a2-4f03-8e2b-196568d1b6ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:20.268708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e0502-10a2-4f03-8e2b-196568d1b6ed None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:20.285782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 295/592] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:20.285782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e0502-10a2-4f03-8e2b-196568d1b6ed None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:20.338830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beeffaf-41b6-491f-83be-f6c40d35b7ff None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:20.371612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:20.373817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11685f8b-a008-4bd0-88b7-f3414689d9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:20.374443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11685f8b-a008-4bd0-88b7-f3414689d9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:20.374718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11685f8b-a008-4bd0-88b7-f3414689d9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:20.374984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11685f8b-a008-4bd0-88b7-f3414689d9d7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:20.382672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11685f8b-a008-4bd0-88b7-f3414689d9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:20.395269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11685f8b-a008-4bd0-88b7-f3414689d9d7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:20.395269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:20.395468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e0502-10a2-4f03-8e2b-196568d1b6ed None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-11685f8b-a008-4bd0-88b7-f3414689d9d7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:20.396328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:20.397238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 298/593] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:20.398551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11685f8b-a008-4bd0-88b7-f3414689d9d7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:20.427544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c30b3-28f1-4b1a-aaf6-cdb9d6ca3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:20.428207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c30b3-28f1-4b1a-aaf6-cdb9d6ca3806 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:20.428492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c30b3-28f1-4b1a-aaf6-cdb9d6ca3806 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:20.428778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c30b3-28f1-4b1a-aaf6-cdb9d6ca3806 None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/9901608b62344115aa561d00bfe88c1c/roles/e33e2d7e93b4409fb2c449241a219d0d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:20.443785 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c30b3-28f1-4b1a-aaf6-cdb9d6ca3806 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=9901608b62344115aa561d00bfe88c1c, role_id=e33e2d7e93b4409fb2c449241a219d0d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:20.445226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c30b3-28f1-4b1a-aaf6-cdb9d6ca3806 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:20.504967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11685f8b-a008-4bd0-88b7-f3414689d9d7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:20.517217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 296/594] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:30:20 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/9901608b62344115aa561d00bfe88c1c/roles/e33e2d7e93b4409fb2c449241a219d0d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:30:20.518542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c30b3-28f1-4b1a-aaf6-cdb9d6ca3806 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:20.542912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381f31e-1f0a-4343-ba9a-1a074e052a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:20.543492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381f31e-1f0a-4343-ba9a-1a074e052a64 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:20.543783 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381f31e-1f0a-4343-ba9a-1a074e052a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:20.544139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381f31e-1f0a-4343-ba9a-1a074e052a64 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:20.545072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381f31e-1f0a-4343-ba9a-1a074e052a64 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:20.545377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381f31e-1f0a-4343-ba9a-1a074e052a64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:20.546950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381f31e-1f0a-4343-ba9a-1a074e052a64 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:20.566900 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 299/595] 10.222.0.42 () {56 vars in 1139 bytes} [Sat Sep 17 03:30:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:20.573064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381f31e-1f0a-4343-ba9a-1a074e052a64 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:20.647600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:20.682205 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c30b3-28f1-4b1a-aaf6-cdb9d6ca3806 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:20.707175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381f31e-1f0a-4343-ba9a-1a074e052a64 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:20.720649 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e3889-8c05-4560-a3ef-73852b25d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:20.721321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e3889-8c05-4560-a3ef-73852b25d2bb None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:20.721606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e3889-8c05-4560-a3ef-73852b25d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:20.722192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e3889-8c05-4560-a3ef-73852b25d2bb None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/ab54ee52004746eaadb5c2970e59afc3/roles/e33e2d7e93b4409fb2c449241a219d0d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:20.724475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:20.739317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e3889-8c05-4560-a3ef-73852b25d2bb None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=ab54ee52004746eaadb5c2970e59afc3, role_id=e33e2d7e93b4409fb2c449241a219d0d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:20.741227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e3889-8c05-4560-a3ef-73852b25d2bb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:20.752569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21869961-29af-4e65-af22-64652e4a4e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:20.753887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21869961-29af-4e65-af22-64652e4a4e9e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:20.753887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21869961-29af-4e65-af22-64652e4a4e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:20.753887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21869961-29af-4e65-af22-64652e4a4e9e None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:20.754777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21869961-29af-4e65-af22-64652e4a4e9e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:20.755150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21869961-29af-4e65-af22-64652e4a4e9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:20.756940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21869961-29af-4e65-af22-64652e4a4e9e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:20.778134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 300/596] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Sep 17 03:30:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:20.781914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21869961-29af-4e65-af22-64652e4a4e9e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:20.826202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 297/597] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:30:20 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/ab54ee52004746eaadb5c2970e59afc3/roles/e33e2d7e93b4409fb2c449241a219d0d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:30:20.828517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e3889-8c05-4560-a3ef-73852b25d2bb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:20.981523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e3889-8c05-4560-a3ef-73852b25d2bb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:21.021146 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194886d-d18b-427b-938a-361f74d88d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:21.021810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194886d-d18b-427b-938a-361f74d88d65 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:21.022098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194886d-d18b-427b-938a-361f74d88d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:21.022377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194886d-d18b-427b-938a-361f74d88d65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:21.023293 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21869961-29af-4e65-af22-64652e4a4e9e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:21.035026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194886d-d18b-427b-938a-361f74d88d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:21.036606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194886d-d18b-427b-938a-361f74d88d65 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:21.038834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-6194886d-d18b-427b-938a-361f74d88d65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:21.054331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:21.055346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:21.057887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:21.057887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:21.057887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:21.057887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 298/598] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:21.061876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194886d-d18b-427b-938a-361f74d88d65 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:21.075538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbd647-7034-49f3-9ced-bcb9f94087ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:21.082501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbd647-7034-49f3-9ced-bcb9f94087ae None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:21.082786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbd647-7034-49f3-9ced-bcb9f94087ae None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:21.083067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbd647-7034-49f3-9ced-bcb9f94087ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:21.102904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbd647-7034-49f3-9ced-bcb9f94087ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:21.105220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbd647-7034-49f3-9ced-bcb9f94087ae None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:21.114097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d5bbd647-7034-49f3-9ced-bcb9f94087ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:21.123255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:21.124234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:21.125513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:21.125513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:21.125513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:21.125513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 301/599] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:21.130124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbd647-7034-49f3-9ced-bcb9f94087ae None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:21.172098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194886d-d18b-427b-938a-361f74d88d65 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:21.208786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed343f-089a-4582-a55d-13cc50e85cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:21.209453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed343f-089a-4582-a55d-13cc50e85cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:21.209754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed343f-089a-4582-a55d-13cc50e85cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:21.210042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed343f-089a-4582-a55d-13cc50e85cc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:21.210803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed343f-089a-4582-a55d-13cc50e85cc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:21.211114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed343f-089a-4582-a55d-13cc50e85cc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:21.212589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed343f-089a-4582-a55d-13cc50e85cc5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:21.225868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 299/600] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:21.233010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed343f-089a-4582-a55d-13cc50e85cc5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:21.334314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:21.338623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed343f-089a-4582-a55d-13cc50e85cc5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:21.379892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c0ab5-cadf-4363-9f6b-b9c0fc63442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:21.380553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0ab5-cadf-4363-9f6b-b9c0fc63442f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:21.380844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0ab5-cadf-4363-9f6b-b9c0fc63442f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:21.381194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0ab5-cadf-4363-9f6b-b9c0fc63442f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:21.390885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c0ab5-cadf-4363-9f6b-b9c0fc63442f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:21.392606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c0ab5-cadf-4363-9f6b-b9c0fc63442f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f53c0ab5-cadf-4363-9f6b-b9c0fc63442f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:21.407661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:21.408664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:21.409995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:21.409995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:21.409995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:21.409995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:21.409995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:21.409995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:21.409995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:21.409995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:21.409995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 300/601] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:21.414740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c0ab5-cadf-4363-9f6b-b9c0fc63442f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:21.481057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbd647-7034-49f3-9ced-bcb9f94087ae None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:21.546727 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0327e-99ae-4914-abe3-912e194bbe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:21.547963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0327e-99ae-4914-abe3-912e194bbe7a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:21.548452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0327e-99ae-4914-abe3-912e194bbe7a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:21.549143 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0327e-99ae-4914-abe3-912e194bbe7a None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:21.550681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0327e-99ae-4914-abe3-912e194bbe7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:21.551227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0327e-99ae-4914-abe3-912e194bbe7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:21.552566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c0ab5-cadf-4363-9f6b-b9c0fc63442f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:21.553068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0327e-99ae-4914-abe3-912e194bbe7a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:21.573904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 302/602] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:21.573904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf2b01-7a4b-4f8e-be32-8e27f935eead None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:21.573904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf2b01-7a4b-4f8e-be32-8e27f935eead None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:21.573904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf2b01-7a4b-4f8e-be32-8e27f935eead None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:21.573904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 303/603] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:21.579514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf2b01-7a4b-4f8e-be32-8e27f935eead None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:21.606368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941a82b-124a-41b7-90a3-2e59c1e1e247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:21.607741 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941a82b-124a-41b7-90a3-2e59c1e1e247 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:21.608463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941a82b-124a-41b7-90a3-2e59c1e1e247 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:21.608940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941a82b-124a-41b7-90a3-2e59c1e1e247 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:21.609674 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:21.611143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941a82b-124a-41b7-90a3-2e59c1e1e247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:21.611772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941a82b-124a-41b7-90a3-2e59c1e1e247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:21.613870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941a82b-124a-41b7-90a3-2e59c1e1e247 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:21.632351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 301/604] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:21.634182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8359d4d-aa2f-4618-8be4-4be7cb1a4142 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:21.638019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8359d4d-aa2f-4618-8be4-4be7cb1a4142 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:21.638019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8359d4d-aa2f-4618-8be4-4be7cb1a4142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:21.660058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e8359d4d-aa2f-4618-8be4-4be7cb1a4142 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:21.661813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e8359d4d-aa2f-4618-8be4-4be7cb1a4142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:21.676781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:21.700737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf2b01-7a4b-4f8e-be32-8e27f935eead None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:21.741205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87ea62-5272-470b-8b03-d5ef453773eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:21.741974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87ea62-5272-470b-8b03-d5ef453773eb None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:21.742329 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87ea62-5272-470b-8b03-d5ef453773eb None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:21.742758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87ea62-5272-470b-8b03-d5ef453773eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:21.748870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 302/605] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:21.749998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8359d4d-aa2f-4618-8be4-4be7cb1a4142 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:21.756554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87ea62-5272-470b-8b03-d5ef453773eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:21.758063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87ea62-5272-470b-8b03-d5ef453773eb None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-9a87ea62-5272-470b-8b03-d5ef453773eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:21.778080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:21.779018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:21.779964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:21.779964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:21.779964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:21.779964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:21.779964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:21.779964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:21.779964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:21.779964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:21.790533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 304/606] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:21.807620 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87ea62-5272-470b-8b03-d5ef453773eb None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:21.863639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8359d4d-aa2f-4618-8be4-4be7cb1a4142 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:21.898275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508c049-6174-4312-935b-423c5a4f73e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:21.899407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508c049-6174-4312-935b-423c5a4f73e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:21.899906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508c049-6174-4312-935b-423c5a4f73e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:21.900480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508c049-6174-4312-935b-423c5a4f73e6 None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/727f7207d82640a3a228dfb83814148b/roles/e33e2d7e93b4409fb2c449241a219d0d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:21.915327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508c049-6174-4312-935b-423c5a4f73e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=727f7207d82640a3a228dfb83814148b, role_id=e33e2d7e93b4409fb2c449241a219d0d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:21.917571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508c049-6174-4312-935b-423c5a4f73e6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:21.921239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:21.968655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:21.984634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87ea62-5272-470b-8b03-d5ef453773eb None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:22.024668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 303/607] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:30:21 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/727f7207d82640a3a228dfb83814148b/roles/e33e2d7e93b4409fb2c449241a219d0d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:30:22.026803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e60da-ee3c-42d3-949f-3378a9289828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:22.027415 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e60da-ee3c-42d3-949f-3378a9289828 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:22.027691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e60da-ee3c-42d3-949f-3378a9289828 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:22.027996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e60da-ee3c-42d3-949f-3378a9289828 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:22.028758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e60da-ee3c-42d3-949f-3378a9289828 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:22.029102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e60da-ee3c-42d3-949f-3378a9289828 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:22.030730 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e60da-ee3c-42d3-949f-3378a9289828 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:22.049865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 305/608] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:22.052648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508c049-6174-4312-935b-423c5a4f73e6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:22.165109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508c049-6174-4312-935b-423c5a4f73e6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:22.175319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e60da-ee3c-42d3-949f-3378a9289828 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:22.223878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88489813-622b-4c8c-994a-dcc2af9b1ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:22.224562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88489813-622b-4c8c-994a-dcc2af9b1ae1 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:22.224838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88489813-622b-4c8c-994a-dcc2af9b1ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:22.225158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88489813-622b-4c8c-994a-dcc2af9b1ae1 None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/6336f0c469b84db78a90cf8435c7dbe4/roles/e33e2d7e93b4409fb2c449241a219d0d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:22.251928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88489813-622b-4c8c-994a-dcc2af9b1ae1 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=6336f0c469b84db78a90cf8435c7dbe4, role_id=e33e2d7e93b4409fb2c449241a219d0d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:22.253458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88489813-622b-4c8c-994a-dcc2af9b1ae1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:22.296234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e60da-ee3c-42d3-949f-3378a9289828 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:22.332929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d85641-d9ba-408a-8caa-3bbd570fc8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:22.333564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d85641-d9ba-408a-8caa-3bbd570fc8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:22.333877 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d85641-d9ba-408a-8caa-3bbd570fc8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:22.334249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d85641-d9ba-408a-8caa-3bbd570fc8f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:22.338405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 304/609] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:30:22 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/6336f0c469b84db78a90cf8435c7dbe4/roles/e33e2d7e93b4409fb2c449241a219d0d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:30:22.346523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d85641-d9ba-408a-8caa-3bbd570fc8f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-52d85641-d9ba-408a-8caa-3bbd570fc8f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:22.351508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:22.352554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:22.353709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 306/610] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:22.359184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88489813-622b-4c8c-994a-dcc2af9b1ae1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:22.417652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:22.477262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88489813-622b-4c8c-994a-dcc2af9b1ae1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:22.514078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14932af4-24b4-4647-81c8-b5ff411d664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:22.514700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14932af4-24b4-4647-81c8-b5ff411d664f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:22.514983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14932af4-24b4-4647-81c8-b5ff411d664f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:22.515261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14932af4-24b4-4647-81c8-b5ff411d664f None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:22.515950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14932af4-24b4-4647-81c8-b5ff411d664f None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:22.516259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14932af4-24b4-4647-81c8-b5ff411d664f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:22.517659 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14932af4-24b4-4647-81c8-b5ff411d664f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:22.528143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 305/611] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:22.534038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d85641-d9ba-408a-8caa-3bbd570fc8f1 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:22.544522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:22.645822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d85641-d9ba-408a-8caa-3bbd570fc8f1 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:22.682684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120de155-e431-4bfa-bc94-a6fe4e77f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:22.683252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120de155-e431-4bfa-bc94-a6fe4e77f0ec None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:22.683529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120de155-e431-4bfa-bc94-a6fe4e77f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:22.683801 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120de155-e431-4bfa-bc94-a6fe4e77f0ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:22.694587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120de155-e431-4bfa-bc94-a6fe4e77f0ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:22.696113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120de155-e431-4bfa-bc94-a6fe4e77f0ec None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-120de155-e431-4bfa-bc94-a6fe4e77f0ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:22.712264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:22.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:22.714540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:22.714540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:22.714540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:22.714540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 307/612] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:22.720570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14932af4-24b4-4647-81c8-b5ff411d664f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:22.859309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14932af4-24b4-4647-81c8-b5ff411d664f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:22.873072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:22.903597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929e063-39c6-406e-9659-11acc294b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:22.904213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929e063-39c6-406e-9659-11acc294b25a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:22.904494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929e063-39c6-406e-9659-11acc294b25a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:22.904773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929e063-39c6-406e-9659-11acc294b25a None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:22.905424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929e063-39c6-406e-9659-11acc294b25a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:22.905743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929e063-39c6-406e-9659-11acc294b25a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:22.907133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929e063-39c6-406e-9659-11acc294b25a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:22.925007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 306/613] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:22.930451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120de155-e431-4bfa-bc94-a6fe4e77f0ec None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:23.018236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:23.071855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2277ce-6246-4d5e-b9d9-166c14896681 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:23.072240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2277ce-6246-4d5e-b9d9-166c14896681 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:23.072692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2277ce-6246-4d5e-b9d9-166c14896681 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:23.075757 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 307/614] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:23.078919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120de155-e431-4bfa-bc94-a6fe4e77f0ec None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:23.079709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8202f-2f33-47d1-88eb-c0f6e166118b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:23.079991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8202f-2f33-47d1-88eb-c0f6e166118b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:23.080263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8202f-2f33-47d1-88eb-c0f6e166118b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:23.099633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c1f8202f-2f33-47d1-88eb-c0f6e166118b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:23.100837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c1f8202f-2f33-47d1-88eb-c0f6e166118b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:23.155901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b3915-727b-490a-a5c8-840115b1cd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:23.156531 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b3915-727b-490a-a5c8-840115b1cd33 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:23.156802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b3915-727b-490a-a5c8-840115b1cd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:23.157066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b3915-727b-490a-a5c8-840115b1cd33 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:23.168737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b3915-727b-490a-a5c8-840115b1cd33 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:23.170215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b3915-727b-490a-a5c8-840115b1cd33 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-937b3915-727b-490a-a5c8-840115b1cd33 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:23.187602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:23.188600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:23.189556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 308/615] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:23.196053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b3915-727b-490a-a5c8-840115b1cd33 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:23.210295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 308/616] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:30:23.356963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b3915-727b-490a-a5c8-840115b1cd33 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:23.381024 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:23.404771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baa1db-7d1c-4d45-a119-88a18d1ff52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:23.405387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baa1db-7d1c-4d45-a119-88a18d1ff52a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:23.405668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baa1db-7d1c-4d45-a119-88a18d1ff52a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:23.405972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baa1db-7d1c-4d45-a119-88a18d1ff52a None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:23.406711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baa1db-7d1c-4d45-a119-88a18d1ff52a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:23.407019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baa1db-7d1c-4d45-a119-88a18d1ff52a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:23.408579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baa1db-7d1c-4d45-a119-88a18d1ff52a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:23.434465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 309/617] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Sep 17 03:30:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:23.440611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8202f-2f33-47d1-88eb-c0f6e166118b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:23.553029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:23.558481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8202f-2f33-47d1-88eb-c0f6e166118b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:23.564231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baa1db-7d1c-4d45-a119-88a18d1ff52a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:23.598852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccd691-e77e-4d40-8864-1ac387a23183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:23.599904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccd691-e77e-4d40-8864-1ac387a23183 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:23.600404 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccd691-e77e-4d40-8864-1ac387a23183 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:23.600889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccd691-e77e-4d40-8864-1ac387a23183 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:23.612657 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccd691-e77e-4d40-8864-1ac387a23183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:23.614561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccd691-e77e-4d40-8864-1ac387a23183 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-48ccd691-e77e-4d40-8864-1ac387a23183 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:23.632307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:23.633405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:23.635199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:23.635199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:23.635199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:23.637618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 309/618] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:23.639122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af42a3-6993-4e06-a56a-eedf410cc251 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:23.639713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af42a3-6993-4e06-a56a-eedf410cc251 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:23.640200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af42a3-6993-4e06-a56a-eedf410cc251 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:23.641938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 310/619] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:23.642911 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af42a3-6993-4e06-a56a-eedf410cc251 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:23.704259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baa1db-7d1c-4d45-a119-88a18d1ff52a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:23.739227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79ad53-a02e-4c20-9542-33f5b96623fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:23.739849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79ad53-a02e-4c20-9542-33f5b96623fc None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:23.740121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79ad53-a02e-4c20-9542-33f5b96623fc None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:23.740388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79ad53-a02e-4c20-9542-33f5b96623fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:23.751438 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79ad53-a02e-4c20-9542-33f5b96623fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-ab79ad53-a02e-4c20-9542-33f5b96623fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:23.756271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:23.757239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:23.758159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:23.762269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 310/620] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:23.762847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af42a3-6993-4e06-a56a-eedf410cc251 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:23.763521 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4767b3a-fafa-46f3-bb2c-0ab578ce402d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:23.763805 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4767b3a-fafa-46f3-bb2c-0ab578ce402d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:23.764079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4767b3a-fafa-46f3-bb2c-0ab578ce402d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:23.782628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-c4767b3a-fafa-46f3-bb2c-0ab578ce402d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:23.783807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c4767b3a-fafa-46f3-bb2c-0ab578ce402d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:23.811485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65616fef-133e-45c7-a643-d036ddec57b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:23.812359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65616fef-133e-45c7-a643-d036ddec57b1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:23.812853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65616fef-133e-45c7-a643-d036ddec57b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:23.813342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65616fef-133e-45c7-a643-d036ddec57b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:23.813963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:23.814512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65616fef-133e-45c7-a643-d036ddec57b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:23.815017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65616fef-133e-45c7-a643-d036ddec57b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:23.816688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65616fef-133e-45c7-a643-d036ddec57b1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:23.835005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 311/621] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:23.836163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65616fef-133e-45c7-a643-d036ddec57b1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:23.861488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:23.864212 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 311/622] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:23.865645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4767b3a-fafa-46f3-bb2c-0ab578ce402d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:23.952801 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65616fef-133e-45c7-a643-d036ddec57b1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:24.014648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4767b3a-fafa-46f3-bb2c-0ab578ce402d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:24.016109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb272a-9542-48ab-b68b-aebd98968b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:24.016935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb272a-9542-48ab-b68b-aebd98968b1c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.018799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb272a-9542-48ab-b68b-aebd98968b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.019277 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb272a-9542-48ab-b68b-aebd98968b1c None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.020131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb272a-9542-48ab-b68b-aebd98968b1c None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:24.020816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb272a-9542-48ab-b68b-aebd98968b1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:24.023194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb272a-9542-48ab-b68b-aebd98968b1c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:24.040088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 312/623] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:24.046227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb272a-9542-48ab-b68b-aebd98968b1c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:24.088576 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdeb21c-e70f-43cd-97d5-7229d9847507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:24.089517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdeb21c-e70f-43cd-97d5-7229d9847507 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.090024 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdeb21c-e70f-43cd-97d5-7229d9847507 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.091547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdeb21c-e70f-43cd-97d5-7229d9847507 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.106657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdeb21c-e70f-43cd-97d5-7229d9847507 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:24.110171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdeb21c-e70f-43cd-97d5-7229d9847507 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-1cdeb21c-e70f-43cd-97d5-7229d9847507 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:24.135751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:24.136720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:24.137650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:24.137650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:24.137650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:24.137650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:24.137650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:24.137650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:24.137650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:24.137650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:24.143846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 312/624] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:24.149357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdeb21c-e70f-43cd-97d5-7229d9847507 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:24.189987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:24.196961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:24.220368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb272a-9542-48ab-b68b-aebd98968b1c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:24.257046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bba60-daab-4d55-9703-d7b82d99754b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:24.257783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bba60-daab-4d55-9703-d7b82d99754b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.258134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bba60-daab-4d55-9703-d7b82d99754b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.258482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bba60-daab-4d55-9703-d7b82d99754b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.271200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73bba60-daab-4d55-9703-d7b82d99754b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:24.272803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73bba60-daab-4d55-9703-d7b82d99754b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d73bba60-daab-4d55-9703-d7b82d99754b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:24.287994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:24.289013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:24.289995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:24.289995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:24.289995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:24.294223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 313/625] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:24.295129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bba60-daab-4d55-9703-d7b82d99754b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:24.297519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdeb21c-e70f-43cd-97d5-7229d9847507 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:24.347469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261653c-0f43-4bdf-81d2-81d694b16c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:24.348593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261653c-0f43-4bdf-81d2-81d694b16c06 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.348867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261653c-0f43-4bdf-81d2-81d694b16c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.349133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261653c-0f43-4bdf-81d2-81d694b16c06 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.349850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261653c-0f43-4bdf-81d2-81d694b16c06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:24.350605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261653c-0f43-4bdf-81d2-81d694b16c06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:24.352786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261653c-0f43-4bdf-81d2-81d694b16c06 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:24.369423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 313/626] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:24.370469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261653c-0f43-4bdf-81d2-81d694b16c06 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:24.423330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bba60-daab-4d55-9703-d7b82d99754b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:24.451221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:24.463242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4508b-5a0b-4217-aa4b-dd35086f3435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:24.464219 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4508b-5a0b-4217-aa4b-dd35086f3435 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.464744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4508b-5a0b-4217-aa4b-dd35086f3435 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.464973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4508b-5a0b-4217-aa4b-dd35086f3435 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.477305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4508b-5a0b-4217-aa4b-dd35086f3435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-7af4508b-5a0b-4217-aa4b-dd35086f3435 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:24.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:24.484302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:24.485363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:24.486550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 314/627] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:24.491400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:24.492293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4508b-5a0b-4217-aa4b-dd35086f3435 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:24.507269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261653c-0f43-4bdf-81d2-81d694b16c06 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:24.550470 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de808db-8289-4071-a5f8-fe722f18222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:24.551075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de808db-8289-4071-a5f8-fe722f18222b None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.551351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de808db-8289-4071-a5f8-fe722f18222b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.551618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de808db-8289-4071-a5f8-fe722f18222b None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.552259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de808db-8289-4071-a5f8-fe722f18222b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:24.552555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de808db-8289-4071-a5f8-fe722f18222b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:24.554589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de808db-8289-4071-a5f8-fe722f18222b None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:24.570987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 314/628] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:24.572073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de808db-8289-4071-a5f8-fe722f18222b None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:24.616387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4508b-5a0b-4217-aa4b-dd35086f3435 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:24.657143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee670d-79d7-4d8d-934c-88b71677119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:24.658553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee670d-79d7-4d8d-934c-88b71677119e None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.659056 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee670d-79d7-4d8d-934c-88b71677119e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.659896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee670d-79d7-4d8d-934c-88b71677119e None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/c1136770f79a47f6a2af0b0a32259209/roles/e33e2d7e93b4409fb2c449241a219d0d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.680777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee670d-79d7-4d8d-934c-88b71677119e None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=c1136770f79a47f6a2af0b0a32259209, role_id=e33e2d7e93b4409fb2c449241a219d0d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:24.684166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee670d-79d7-4d8d-934c-88b71677119e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:24.694940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de808db-8289-4071-a5f8-fe722f18222b None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:24.728721 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b15d6-cec4-4d2f-9436-e24baa59114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:24.729324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b15d6-cec4-4d2f-9436-e24baa59114d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.729602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b15d6-cec4-4d2f-9436-e24baa59114d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.729931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b15d6-cec4-4d2f-9436-e24baa59114d None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.730597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b15d6-cec4-4d2f-9436-e24baa59114d None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:24.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b15d6-cec4-4d2f-9436-e24baa59114d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:24.732325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b15d6-cec4-4d2f-9436-e24baa59114d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:24.740299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 315/629] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:24.741078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b15d6-cec4-4d2f-9436-e24baa59114d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:24.750685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 315/630] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:30:24 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/c1136770f79a47f6a2af0b0a32259209/roles/e33e2d7e93b4409fb2c449241a219d0d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:30:24.751928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244c89a-9308-4565-9959-ed9d3049c84b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.752219 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244c89a-9308-4565-9959-ed9d3049c84b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.752501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244c89a-9308-4565-9959-ed9d3049c84b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.757982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 316/631] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:24.759208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244c89a-9308-4565-9959-ed9d3049c84b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:24.764797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:24.808168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:24.882219 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244c89a-9308-4565-9959-ed9d3049c84b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:24.903079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b15d6-cec4-4d2f-9436-e24baa59114d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:24.925302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69179e-bbf6-4db1-8402-98a4a7cc1b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:24.926066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69179e-bbf6-4db1-8402-98a4a7cc1b95 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.926418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69179e-bbf6-4db1-8402-98a4a7cc1b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.926762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69179e-bbf6-4db1-8402-98a4a7cc1b95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.940225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69179e-bbf6-4db1-8402-98a4a7cc1b95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:24.941925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69179e-bbf6-4db1-8402-98a4a7cc1b95 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:24.959308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfa13f-cf66-4a73-bd34-49791e091e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:24.961069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa13f-cf66-4a73-bd34-49791e091e65 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-8c69179e-bbf6-4db1-8402-98a4a7cc1b95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:24.961166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:24.962349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:24.964079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:24.964079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:24.964079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:24.964285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa13f-cf66-4a73-bd34-49791e091e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.964285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa13f-cf66-4a73-bd34-49791e091e65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.964440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 317/632] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:24.965657 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f94fb-5f29-4f9f-b1cc-8538ac2decd4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:24.965657 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f94fb-5f29-4f9f-b1cc-8538ac2decd4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:24.965657 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f94fb-5f29-4f9f-b1cc-8538ac2decd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:24.974332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfa13f-cf66-4a73-bd34-49791e091e65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:24.975859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfa13f-cf66-4a73-bd34-49791e091e65 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:24.986298 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-da6f94fb-5f29-4f9f-b1cc-8538ac2decd4 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:24.987495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-da6f94fb-5f29-4f9f-b1cc-8538ac2decd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-3ebfa13f-cf66-4a73-bd34-49791e091e65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:24.991894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:24.992854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 316/633] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Sep 17 03:30:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:24.994973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfa13f-cf66-4a73-bd34-49791e091e65 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:25.063389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 318/634] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:25.064051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f94fb-5f29-4f9f-b1cc-8538ac2decd4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:25.076691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:25.083917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:25.113160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfa13f-cf66-4a73-bd34-49791e091e65 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:25.149832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eda7b6-424a-4fd3-adb4-ede717bfd224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:25.151604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eda7b6-424a-4fd3-adb4-ede717bfd224 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.151870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eda7b6-424a-4fd3-adb4-ede717bfd224 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.152138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eda7b6-424a-4fd3-adb4-ede717bfd224 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:25.152786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eda7b6-424a-4fd3-adb4-ede717bfd224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:25.153092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eda7b6-424a-4fd3-adb4-ede717bfd224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:25.154623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eda7b6-424a-4fd3-adb4-ede717bfd224 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:25.172008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 317/635] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:25.172008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe3f51-093f-44e6-9e5a-40387a4bdfe9 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.172008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe3f51-093f-44e6-9e5a-40387a4bdfe9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.172008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe3f51-093f-44e6-9e5a-40387a4bdfe9 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:25.172008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 318/636] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:25.179026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ab06f-b438-4023-b660-caab15fbc4d8 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.179314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ab06f-b438-4023-b660-caab15fbc4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.179590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ab06f-b438-4023-b660-caab15fbc4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:25.180114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f94fb-5f29-4f9f-b1cc-8538ac2decd4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:25.199242 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-189ab06f-b438-4023-b660-caab15fbc4d8 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:25.200243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-189ab06f-b438-4023-b660-caab15fbc4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:25.217017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cae3d-eacf-4127-a4fb-5f1868ce5693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:25.217938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cae3d-eacf-4127-a4fb-5f1868ce5693 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.218381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cae3d-eacf-4127-a4fb-5f1868ce5693 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.218852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cae3d-eacf-4127-a4fb-5f1868ce5693 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:25.219718 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cae3d-eacf-4127-a4fb-5f1868ce5693 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:25.220161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cae3d-eacf-4127-a4fb-5f1868ce5693 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:25.221862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cae3d-eacf-4127-a4fb-5f1868ce5693 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:25.246380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 319/637] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Sep 17 03:30:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:25.247507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cae3d-eacf-4127-a4fb-5f1868ce5693 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:25.278230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 319/638] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:25.279179 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ab06f-b438-4023-b660-caab15fbc4d8 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:25.338475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:25.379796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cae3d-eacf-4127-a4fb-5f1868ce5693 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:25.382080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:25.434148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ab06f-b438-4023-b660-caab15fbc4d8 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:25.435399 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62801f23-edb3-4127-b1cc-601fc9c94002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:25.436386 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62801f23-edb3-4127-b1cc-601fc9c94002 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.436765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62801f23-edb3-4127-b1cc-601fc9c94002 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.437186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62801f23-edb3-4127-b1cc-601fc9c94002 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:25.447207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62801f23-edb3-4127-b1cc-601fc9c94002 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:25.450099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62801f23-edb3-4127-b1cc-601fc9c94002 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-62801f23-edb3-4127-b1cc-601fc9c94002 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:25.465654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:25.467179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:25.468121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:25.468121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:25.468121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:25.468121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:25.468121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:25.468121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:25.468121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:25.468121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:25.468121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 03:30:25.468121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:25.470115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 320/639] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:25.471141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62801f23-edb3-4127-b1cc-601fc9c94002 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:25.482055 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51989bf-76c9-4359-b6b6-17783793a371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:25.482665 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51989bf-76c9-4359-b6b6-17783793a371 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.482939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51989bf-76c9-4359-b6b6-17783793a371 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.483209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51989bf-76c9-4359-b6b6-17783793a371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:25.554490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51989bf-76c9-4359-b6b6-17783793a371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:25.555991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51989bf-76c9-4359-b6b6-17783793a371 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:25.583731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62801f23-edb3-4127-b1cc-601fc9c94002 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:25.616277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:25.619823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a799eb-00b8-405b-b5ee-4b07f3d2c5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:25.620603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a799eb-00b8-405b-b5ee-4b07f3d2c5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.620888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a799eb-00b8-405b-b5ee-4b07f3d2c5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.621171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a799eb-00b8-405b-b5ee-4b07f3d2c5e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:25.623072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:25.632260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a799eb-00b8-405b-b5ee-4b07f3d2c5e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-89a799eb-00b8-405b-b5ee-4b07f3d2c5e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:25.637502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:25.638682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:25.639904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:25.643310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 321/640] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:25.644130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a799eb-00b8-405b-b5ee-4b07f3d2c5e9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:25.668402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 320/641] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:25.673742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51989bf-76c9-4359-b6b6-17783793a371 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:25.754368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a799eb-00b8-405b-b5ee-4b07f3d2c5e9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:25.781488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51989bf-76c9-4359-b6b6-17783793a371 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:25.791540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41da7-91da-41e3-824a-00b165682a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:25.792468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41da7-91da-41e3-824a-00b165682a30 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.792762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41da7-91da-41e3-824a-00b165682a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.793197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41da7-91da-41e3-824a-00b165682a30 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:25.793992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e41da7-91da-41e3-824a-00b165682a30 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:25.794310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e41da7-91da-41e3-824a-00b165682a30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:25.795762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e41da7-91da-41e3-824a-00b165682a30 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:25.814288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 322/642] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Sep 17 03:30:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:25.818876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41da7-91da-41e3-824a-00b165682a30 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:25.822462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6007fd-aaf8-4483-9562-72280aafb9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:25.823254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6007fd-aaf8-4483-9562-72280aafb9ec None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.823852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6007fd-aaf8-4483-9562-72280aafb9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.824132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6007fd-aaf8-4483-9562-72280aafb9ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:25.834433 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6007fd-aaf8-4483-9562-72280aafb9ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-fc6007fd-aaf8-4483-9562-72280aafb9ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:25.839953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:25.840950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:25.841887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 321/643] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:25.842607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6007fd-aaf8-4483-9562-72280aafb9ec None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:25.862632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:25.881430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:25.929531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41da7-91da-41e3-824a-00b165682a30 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:25.948082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6007fd-aaf8-4483-9562-72280aafb9ec None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:25.963378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67d4da-cbdc-4979-83db-754bb295cf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:25.964175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67d4da-cbdc-4979-83db-754bb295cf51 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.964548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67d4da-cbdc-4979-83db-754bb295cf51 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.964926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67d4da-cbdc-4979-83db-754bb295cf51 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:25.965584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67d4da-cbdc-4979-83db-754bb295cf51 None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:25.965918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67d4da-cbdc-4979-83db-754bb295cf51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:25.967284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67d4da-cbdc-4979-83db-754bb295cf51 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:25.979787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 323/644] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:25.979787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67d4da-cbdc-4979-83db-754bb295cf51 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:25.990114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47b44d-4194-4b59-b7a7-6e9858f56919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:25.990744 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b44d-4194-4b59-b7a7-6e9858f56919 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:25.991023 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b44d-4194-4b59-b7a7-6e9858f56919 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:25.991294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b44d-4194-4b59-b7a7-6e9858f56919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:26.002577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47b44d-4194-4b59-b7a7-6e9858f56919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:26.004323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47b44d-4194-4b59-b7a7-6e9858f56919 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-1b47b44d-4194-4b59-b7a7-6e9858f56919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:26.020311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:26.021476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:26.022799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:26.022799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:26.022799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:26.022799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 322/645] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:26.026627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47b44d-4194-4b59-b7a7-6e9858f56919 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:26.086692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67d4da-cbdc-4979-83db-754bb295cf51 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:26.098410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:26.101177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:26.121310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e739c5-7bdb-41e1-9369-3e07e6f7d95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:26.122013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e739c5-7bdb-41e1-9369-3e07e6f7d95e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:26.122297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e739c5-7bdb-41e1-9369-3e07e6f7d95e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:26.122574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e739c5-7bdb-41e1-9369-3e07e6f7d95e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:26.132419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47b44d-4194-4b59-b7a7-6e9858f56919 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:26.169207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c50d2-69d7-44bc-aca7-37378a3c6c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:26.169841 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c50d2-69d7-44bc-aca7-37378a3c6c91 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:26.170121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c50d2-69d7-44bc-aca7-37378a3c6c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:26.170395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c50d2-69d7-44bc-aca7-37378a3c6c91 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:26.171061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c50d2-69d7-44bc-aca7-37378a3c6c91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:26.171365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c50d2-69d7-44bc-aca7-37378a3c6c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:26.172738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c50d2-69d7-44bc-aca7-37378a3c6c91 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:26.181113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 323/646] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:26.182784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c50d2-69d7-44bc-aca7-37378a3c6c91 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:26.199687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e739c5-7bdb-41e1-9369-3e07e6f7d95e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:26.201264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e739c5-7bdb-41e1-9369-3e07e6f7d95e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:26.287406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c50d2-69d7-44bc-aca7-37378a3c6c91 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:26.306761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 324/647] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:26.307889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e739c5-7bdb-41e1-9369-3e07e6f7d95e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:26.324541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:26.325933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:26.329213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ea7c8-d4eb-4d89-9bb0-c5ac10c10c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:26.329882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ea7c8-d4eb-4d89-9bb0-c5ac10c10c70 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:26.330163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ea7c8-d4eb-4d89-9bb0-c5ac10c10c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:26.330433 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ea7c8-d4eb-4d89-9bb0-c5ac10c10c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:26.341624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ea7c8-d4eb-4d89-9bb0-c5ac10c10c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:26.343142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ea7c8-d4eb-4d89-9bb0-c5ac10c10c70 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-656ea7c8-d4eb-4d89-9bb0-c5ac10c10c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:26.360082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:26.361072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:26.362037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:26.362037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:26.362037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:26.362841 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 324/648] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:26.363164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ea7c8-d4eb-4d89-9bb0-c5ac10c10c70 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:26.420435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e739c5-7bdb-41e1-9369-3e07e6f7d95e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:26.479751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ea7c8-d4eb-4d89-9bb0-c5ac10c10c70 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:26.483899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b8eb1-d961-4e17-a2ab-3c75565a2fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:26.484542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b8eb1-d961-4e17-a2ab-3c75565a2fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:26.484803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b8eb1-d961-4e17-a2ab-3c75565a2fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:26.485081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b8eb1-d961-4e17-a2ab-3c75565a2fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:26.485741 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b8eb1-d961-4e17-a2ab-3c75565a2fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:26.486057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b8eb1-d961-4e17-a2ab-3c75565a2fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:26.487858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b8eb1-d961-4e17-a2ab-3c75565a2fe0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:26.501337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 325/649] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:26.501337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b8eb1-d961-4e17-a2ab-3c75565a2fe0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:26.519665 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79df656-be80-438e-8092-1bfd2d97336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:26.520299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79df656-be80-438e-8092-1bfd2d97336d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:26.520604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79df656-be80-438e-8092-1bfd2d97336d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:26.520845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79df656-be80-438e-8092-1bfd2d97336d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:26.532356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79df656-be80-438e-8092-1bfd2d97336d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:26.534373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79df656-be80-438e-8092-1bfd2d97336d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d79df656-be80-438e-8092-1bfd2d97336d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:26.549639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:26.550824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:26.551759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:26.551759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:26.551759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:26.551759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 325/650] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:26.551759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79df656-be80-438e-8092-1bfd2d97336d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:26.588706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:26.592215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:26.617359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b8eb1-d961-4e17-a2ab-3c75565a2fe0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:26.652770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3ab42-acf1-43db-828c-47a407c9ee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:26.653489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3ab42-acf1-43db-828c-47a407c9ee14 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:26.653800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3ab42-acf1-43db-828c-47a407c9ee14 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:26.654075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3ab42-acf1-43db-828c-47a407c9ee14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:26.659043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79df656-be80-438e-8092-1bfd2d97336d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:26.664852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3ab42-acf1-43db-828c-47a407c9ee14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:26.667157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3ab42-acf1-43db-828c-47a407c9ee14 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f0e3ab42-acf1-43db-828c-47a407c9ee14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:26.681876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:26.682886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:26.683863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:26.683863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:26.683863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:26.687686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 326/651] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:26.688408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3ab42-acf1-43db-828c-47a407c9ee14 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:26.699361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c664472-6602-4d0d-813a-3e49f59afd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:26.699965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c664472-6602-4d0d-813a-3e49f59afd11 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:26.700610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c664472-6602-4d0d-813a-3e49f59afd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:26.700894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c664472-6602-4d0d-813a-3e49f59afd11 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:26.701885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c664472-6602-4d0d-813a-3e49f59afd11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:26.702196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c664472-6602-4d0d-813a-3e49f59afd11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:26.703743 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c664472-6602-4d0d-813a-3e49f59afd11 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:26.717595 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 326/652] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:26.722770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c664472-6602-4d0d-813a-3e49f59afd11 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:26.814261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3ab42-acf1-43db-828c-47a407c9ee14 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:26.832949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:26.838766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c664472-6602-4d0d-813a-3e49f59afd11 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:26.842819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:26.851051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9a625-5003-4d0c-ae69-3ca0051eb334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:26.851691 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9a625-5003-4d0c-ae69-3ca0051eb334 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:26.851973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9a625-5003-4d0c-ae69-3ca0051eb334 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:26.852333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9a625-5003-4d0c-ae69-3ca0051eb334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:26.860472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9a625-5003-4d0c-ae69-3ca0051eb334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:26.861947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9a625-5003-4d0c-ae69-3ca0051eb334 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-24a9a625-5003-4d0c-ae69-3ca0051eb334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:26.873836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:26.880706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:26.881838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:26.881838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:26.881838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:26.881838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:26.881838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:26.881838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:26.881838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:26.881838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:26.882367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b52f3e-5da4-4940-a05a-2e6a6ee468cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:26.882475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 327/653] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:26.882558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b52f3e-5da4-4940-a05a-2e6a6ee468cb None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:26.882558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b52f3e-5da4-4940-a05a-2e6a6ee468cb None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:26.882713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9a625-5003-4d0c-ae69-3ca0051eb334 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:26.882803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b52f3e-5da4-4940-a05a-2e6a6ee468cb None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:26.882803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b52f3e-5da4-4940-a05a-2e6a6ee468cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:26.882803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b52f3e-5da4-4940-a05a-2e6a6ee468cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:26.882803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b52f3e-5da4-4940-a05a-2e6a6ee468cb None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:26.894963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 327/654] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:26.895725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b52f3e-5da4-4940-a05a-2e6a6ee468cb None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:27.000916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9a625-5003-4d0c-ae69-3ca0051eb334 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.007598 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b52f3e-5da4-4940-a05a-2e6a6ee468cb None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.038030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131fabb-2da8-4fff-9754-04132cb7b32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.038754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131fabb-2da8-4fff-9754-04132cb7b32f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.039042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131fabb-2da8-4fff-9754-04132cb7b32f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.039322 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131fabb-2da8-4fff-9754-04132cb7b32f None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.040409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131fabb-2da8-4fff-9754-04132cb7b32f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:27.041768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131fabb-2da8-4fff-9754-04132cb7b32f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.043159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131fabb-2da8-4fff-9754-04132cb7b32f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:27.043450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ca8f1-6162-4038-8111-943adf2ff2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.044080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ca8f1-6162-4038-8111-943adf2ff2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.044360 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ca8f1-6162-4038-8111-943adf2ff2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.044947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ca8f1-6162-4038-8111-943adf2ff2e2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.052791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ca8f1-6162-4038-8111-943adf2ff2e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.054212 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ca8f1-6162-4038-8111-943adf2ff2e2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:27.055723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 328/655] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:27.057744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131fabb-2da8-4fff-9754-04132cb7b32f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d51ca8f1-6162-4038-8111-943adf2ff2e2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:27.066594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:27.067757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:27.068849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:27.068849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:27.068849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:27.068849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:27.068849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:27.068849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:27.068849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:27.068849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:27.068849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 03:30:27.068849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:27.071043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 328/656] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:27.071043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ca8f1-6162-4038-8111-943adf2ff2e2 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:27.089366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:27.107091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:27.160314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131fabb-2da8-4fff-9754-04132cb7b32f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.174366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ca8f1-6162-4038-8111-943adf2ff2e2 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.194636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b3cab-eaa0-4c03-87cc-ca48d42ffc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.195231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b3cab-eaa0-4c03-87cc-ca48d42ffc49 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.195508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b3cab-eaa0-4c03-87cc-ca48d42ffc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.195784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b3cab-eaa0-4c03-87cc-ca48d42ffc49 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.196453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b3cab-eaa0-4c03-87cc-ca48d42ffc49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:27.196756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b3cab-eaa0-4c03-87cc-ca48d42ffc49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.198267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b3cab-eaa0-4c03-87cc-ca48d42ffc49 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:27.208417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dafc91-873b-4cd9-8dc8-84d8379080fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.209037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dafc91-873b-4cd9-8dc8-84d8379080fd None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.209318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dafc91-873b-4cd9-8dc8-84d8379080fd None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.209592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dafc91-873b-4cd9-8dc8-84d8379080fd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.210349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 329/657] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:27.211498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e259f47-a41c-4b62-bc20-740ae07eec24 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.211780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e259f47-a41c-4b62-bc20-740ae07eec24 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.212052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e259f47-a41c-4b62-bc20-740ae07eec24 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.217557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 330/658] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:27.217557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e259f47-a41c-4b62-bc20-740ae07eec24 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:27.217803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dafc91-873b-4cd9-8dc8-84d8379080fd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.218991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dafc91-873b-4cd9-8dc8-84d8379080fd None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-c8dafc91-873b-4cd9-8dc8-84d8379080fd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:27.230875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:27.231894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 329/659] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:27.232937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dafc91-873b-4cd9-8dc8-84d8379080fd None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:27.309187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:27.323174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e259f47-a41c-4b62-bc20-740ae07eec24 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.330762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:27.332503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dafc91-873b-4cd9-8dc8-84d8379080fd None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.357521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a133b4-9c51-4b5e-aafb-bdaf406a7574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.358184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a133b4-9c51-4b5e-aafb-bdaf406a7574 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.358465 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a133b4-9c51-4b5e-aafb-bdaf406a7574 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.358748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a133b4-9c51-4b5e-aafb-bdaf406a7574 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.365246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a133b4-9c51-4b5e-aafb-bdaf406a7574 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.366097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4105431-6ac2-4592-b46b-0930ad70ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.366730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a133b4-9c51-4b5e-aafb-bdaf406a7574 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:27.367062 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4105431-6ac2-4592-b46b-0930ad70ab12 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.367344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4105431-6ac2-4592-b46b-0930ad70ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.367615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4105431-6ac2-4592-b46b-0930ad70ab12 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.368333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4105431-6ac2-4592-b46b-0930ad70ab12 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:27.368637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4105431-6ac2-4592-b46b-0930ad70ab12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.370336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4105431-6ac2-4592-b46b-0930ad70ab12 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-10a133b4-9c51-4b5e-aafb-bdaf406a7574 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:27.378838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:27.379762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:27.380678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:27.380678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:27.380678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:27.380678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:27.380678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:27.380678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:27.380678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:27.380678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:27.380678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 331/660] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:27.381180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a133b4-9c51-4b5e-aafb-bdaf406a7574 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:27.391641 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 330/661] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Sep 17 03:30:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:27.392926 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e843c3-d545-42b5-aa6e-f80533801448 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.393624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e843c3-d545-42b5-aa6e-f80533801448 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.393929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e843c3-d545-42b5-aa6e-f80533801448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.413514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a0e843c3-d545-42b5-aa6e-f80533801448 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:27.414763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a0e843c3-d545-42b5-aa6e-f80533801448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:27.483128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 331/662] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:27.483978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e843c3-d545-42b5-aa6e-f80533801448 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:27.487398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a133b4-9c51-4b5e-aafb-bdaf406a7574 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.524615 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e1b62-38ea-4835-a88d-f00a6f25e9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.525286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e1b62-38ea-4835-a88d-f00a6f25e9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.525567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e1b62-38ea-4835-a88d-f00a6f25e9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.525896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e1b62-38ea-4835-a88d-f00a6f25e9d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.526619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e1b62-38ea-4835-a88d-f00a6f25e9d4 None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=9901608b62344115aa561d00bfe88c1c) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:27.526927 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e1b62-38ea-4835-a88d-f00a6f25e9d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.528355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e1b62-38ea-4835-a88d-f00a6f25e9d4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:27.545764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 332/663] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:27 2022] GET /identity/v3/role_assignments?user.id=9901608b62344115aa561d00bfe88c1c&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:27.545764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e1b62-38ea-4835-a88d-f00a6f25e9d4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:27.562547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:27.597588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:27.618441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e843c3-d545-42b5-aa6e-f80533801448 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.665207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8847803-9b92-475a-ad40-7a956129d808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.666071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8847803-9b92-475a-ad40-7a956129d808 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.666594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8847803-9b92-475a-ad40-7a956129d808 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.673753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8847803-9b92-475a-ad40-7a956129d808 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.673753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8847803-9b92-475a-ad40-7a956129d808 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:27.673753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8847803-9b92-475a-ad40-7a956129d808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.673753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8847803-9b92-475a-ad40-7a956129d808 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:27.681862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e1b62-38ea-4835-a88d-f00a6f25e9d4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.694193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 332/664] 10.222.0.42 () {56 vars in 1139 bytes} [Sat Sep 17 03:30:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:27.695388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8847803-9b92-475a-ad40-7a956129d808 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:27.726607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80e4d1-3675-40cb-9c24-34c87bd1583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.727317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80e4d1-3675-40cb-9c24-34c87bd1583f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.727792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80e4d1-3675-40cb-9c24-34c87bd1583f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.728715 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80e4d1-3675-40cb-9c24-34c87bd1583f None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.729816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80e4d1-3675-40cb-9c24-34c87bd1583f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:27.730199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80e4d1-3675-40cb-9c24-34c87bd1583f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.731704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80e4d1-3675-40cb-9c24-34c87bd1583f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:27.745898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 333/665] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:27.747031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80e4d1-3675-40cb-9c24-34c87bd1583f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:27.824316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8847803-9b92-475a-ad40-7a956129d808 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.848907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:27.860094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:27.861094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfecfe-5af1-499a-bf60-cb92a6241cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.861701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfecfe-5af1-499a-bf60-cb92a6241cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.862007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfecfe-5af1-499a-bf60-cb92a6241cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.862278 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfecfe-5af1-499a-bf60-cb92a6241cd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.870964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80e4d1-3675-40cb-9c24-34c87bd1583f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:27.905598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf219edb-bfcc-458f-b849-5ba2a445505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:27.906249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf219edb-bfcc-458f-b849-5ba2a445505b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:27.906527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf219edb-bfcc-458f-b849-5ba2a445505b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:27.906800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf219edb-bfcc-458f-b849-5ba2a445505b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:27.933895 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfecfe-5af1-499a-bf60-cb92a6241cd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.935598 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfecfe-5af1-499a-bf60-cb92a6241cd9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:27.981877 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf219edb-bfcc-458f-b849-5ba2a445505b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:27.983316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf219edb-bfcc-458f-b849-5ba2a445505b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:28.041741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 333/666] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:28.047165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfecfe-5af1-499a-bf60-cb92a6241cd9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:28.070834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:28.083674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:28.093167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 334/667] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:28.093167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf219edb-bfcc-458f-b849-5ba2a445505b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:28.158453 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfecfe-5af1-499a-bf60-cb92a6241cd9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:28.195464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf219edb-bfcc-458f-b849-5ba2a445505b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:28.199942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5b06a-b6bd-4a3e-b266-6d695c9f1c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:28.200789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5b06a-b6bd-4a3e-b266-6d695c9f1c09 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:28.201269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5b06a-b6bd-4a3e-b266-6d695c9f1c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:28.201838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5b06a-b6bd-4a3e-b266-6d695c9f1c09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:28.202758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5b06a-b6bd-4a3e-b266-6d695c9f1c09 None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=ab54ee52004746eaadb5c2970e59afc3) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:28.203284 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5b06a-b6bd-4a3e-b266-6d695c9f1c09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:28.205417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5b06a-b6bd-4a3e-b266-6d695c9f1c09 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:28.214205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 334/668] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:28 2022] GET /identity/v3/role_assignments?user.id=ab54ee52004746eaadb5c2970e59afc3&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:28.215496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5b06a-b6bd-4a3e-b266-6d695c9f1c09 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:28.232094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a503c5c-ab7a-40cb-ba62-9d38e9094de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:28.232842 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a503c5c-ab7a-40cb-ba62-9d38e9094de2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:28.233195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a503c5c-ab7a-40cb-ba62-9d38e9094de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:28.233543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a503c5c-ab7a-40cb-ba62-9d38e9094de2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:28.245439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a503c5c-ab7a-40cb-ba62-9d38e9094de2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-0a503c5c-ab7a-40cb-ba62-9d38e9094de2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:28.250708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:28.251746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:28.252634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:28.257201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 335/669] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:28.258344 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a503c5c-ab7a-40cb-ba62-9d38e9094de2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:28.320088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:28.371064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:28.381375 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5b06a-b6bd-4a3e-b266-6d695c9f1c09 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:28.383051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a503c5c-ab7a-40cb-ba62-9d38e9094de2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:28.420659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767df386-bb53-4b97-81c3-5a9e426234cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:28.421319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767df386-bb53-4b97-81c3-5a9e426234cf None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:28.421607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767df386-bb53-4b97-81c3-5a9e426234cf None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:28.421942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767df386-bb53-4b97-81c3-5a9e426234cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:28.430267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d739f7-489e-4425-be94-aa0f8c0659dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:28.433354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d739f7-489e-4425-be94-aa0f8c0659dd None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:28.433354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d739f7-489e-4425-be94-aa0f8c0659dd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:28.433354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d739f7-489e-4425-be94-aa0f8c0659dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:28.434462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767df386-bb53-4b97-81c3-5a9e426234cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:28.435987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767df386-bb53-4b97-81c3-5a9e426234cf None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:28.444194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d739f7-489e-4425-be94-aa0f8c0659dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:28.445898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d739f7-489e-4425-be94-aa0f8c0659dd None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-767df386-bb53-4b97-81c3-5a9e426234cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:28.454846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:28.455826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:28.456792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:28.456792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:28.456792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:28.461204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 335/670] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:28.462131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767df386-bb53-4b97-81c3-5a9e426234cf None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-20d739f7-489e-4425-be94-aa0f8c0659dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:28.464915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:28.466050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:28.475019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:28.475019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:28.475019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:28.475019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 336/671] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:28.475019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d739f7-489e-4425-be94-aa0f8c0659dd None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:28.587936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767df386-bb53-4b97-81c3-5a9e426234cf None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:28.605065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d739f7-489e-4425-be94-aa0f8c0659dd None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:28.607234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:28.624238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcb654-1261-4499-a6c1-3ba7a5384f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:28.624923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcb654-1261-4499-a6c1-3ba7a5384f23 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:28.625269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcb654-1261-4499-a6c1-3ba7a5384f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:28.625607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcb654-1261-4499-a6c1-3ba7a5384f23 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:28.626393 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcb654-1261-4499-a6c1-3ba7a5384f23 None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:28.626766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcb654-1261-4499-a6c1-3ba7a5384f23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:28.628193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcb654-1261-4499-a6c1-3ba7a5384f23 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:28.635091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:28.639428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 336/672] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:28.640419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcb654-1261-4499-a6c1-3ba7a5384f23 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:28.648522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773a4d8-bf94-414a-a1df-2fcc4e54b9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:28.649125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773a4d8-bf94-414a-a1df-2fcc4e54b9ee None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:28.649400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773a4d8-bf94-414a-a1df-2fcc4e54b9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:28.650222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773a4d8-bf94-414a-a1df-2fcc4e54b9ee None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:28.650873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773a4d8-bf94-414a-a1df-2fcc4e54b9ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:28.651178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773a4d8-bf94-414a-a1df-2fcc4e54b9ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:28.652991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773a4d8-bf94-414a-a1df-2fcc4e54b9ee None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:28.671252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 337/673] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:28.673065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773a4d8-bf94-414a-a1df-2fcc4e54b9ee None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:28.776349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcb654-1261-4499-a6c1-3ba7a5384f23 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:28.798045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773a4d8-bf94-414a-a1df-2fcc4e54b9ee None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:28.821169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441621c-5599-4414-826c-24fcc4a48e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:28.822446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441621c-5599-4414-826c-24fcc4a48e4e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:28.822846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441621c-5599-4414-826c-24fcc4a48e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:28.823190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441621c-5599-4414-826c-24fcc4a48e4e None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:28.824270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441621c-5599-4414-826c-24fcc4a48e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:28.824562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441621c-5599-4414-826c-24fcc4a48e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:28.826034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441621c-5599-4414-826c-24fcc4a48e4e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:28.845073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 337/674] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:28.845073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441621c-5599-4414-826c-24fcc4a48e4e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:28.849293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0fe12-a65f-41f5-98ce-89a56d89cc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:28.850629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0fe12-a65f-41f5-98ce-89a56d89cc2c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:28.850629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0fe12-a65f-41f5-98ce-89a56d89cc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:28.850629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0fe12-a65f-41f5-98ce-89a56d89cc2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:28.863115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0fe12-a65f-41f5-98ce-89a56d89cc2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:28.865304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0fe12-a65f-41f5-98ce-89a56d89cc2c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-88c0fe12-a65f-41f5-98ce-89a56d89cc2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:28.880030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:28.881017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:28.881966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:28.881966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:28.881966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:28.886604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 338/675] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:28.887550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0fe12-a65f-41f5-98ce-89a56d89cc2c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:28.904373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:28.926478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:28.980426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441621c-5599-4414-826c-24fcc4a48e4e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:28.993267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0fe12-a65f-41f5-98ce-89a56d89cc2c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:29.030261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77f9e5-ede8-45b3-bde5-031af967b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:29.030902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f9e5-ede8-45b3-bde5-031af967b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.031183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f9e5-ede8-45b3-bde5-031af967b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.031457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f9e5-ede8-45b3-bde5-031af967b1f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.033906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcec600-92ba-42df-9d67-53f0ba08a09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:29.034688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcec600-92ba-42df-9d67-53f0ba08a09f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.035083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcec600-92ba-42df-9d67-53f0ba08a09f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.035455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcec600-92ba-42df-9d67-53f0ba08a09f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.038743 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77f9e5-ede8-45b3-bde5-031af967b1f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:29.040182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77f9e5-ede8-45b3-bde5-031af967b1f7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:29.042996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcec600-92ba-42df-9d67-53f0ba08a09f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:29.044553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcec600-92ba-42df-9d67-53f0ba08a09f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-6e77f9e5-ede8-45b3-bde5-031af967b1f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:29.052066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:29.053078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:29.054846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:29.054846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:29.054846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:29.054846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:29.054846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:29.054846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:29.054846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:29.054846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:29.054846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 338/676] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:29.054846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77f9e5-ede8-45b3-bde5-031af967b1f7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4fcec600-92ba-42df-9d67-53f0ba08a09f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:29.057848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:29.060175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:29.062325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:29.062325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:29.062325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:29.062325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:29.062325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:29.062325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:29.062325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:29.062325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:29.065840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 339/677] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:29.065840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcec600-92ba-42df-9d67-53f0ba08a09f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:29.147313 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:29.167148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77f9e5-ede8-45b3-bde5-031af967b1f7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:29.174238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcec600-92ba-42df-9d67-53f0ba08a09f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:29.189588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:29.206387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca8b5e-11a2-457f-b51a-923e1ba39c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:29.207215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca8b5e-11a2-457f-b51a-923e1ba39c0c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.207362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca8b5e-11a2-457f-b51a-923e1ba39c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.207605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca8b5e-11a2-457f-b51a-923e1ba39c0c None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.208254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca8b5e-11a2-457f-b51a-923e1ba39c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:29.208611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca8b5e-11a2-457f-b51a-923e1ba39c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:29.209958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca8b5e-11a2-457f-b51a-923e1ba39c0c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:29.213332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999c856-deb0-40d0-a1d5-62acbdb7f3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:29.214296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999c856-deb0-40d0-a1d5-62acbdb7f3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.214728 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999c856-deb0-40d0-a1d5-62acbdb7f3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.216776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999c856-deb0-40d0-a1d5-62acbdb7f3c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.216776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999c856-deb0-40d0-a1d5-62acbdb7f3c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:29.216776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999c856-deb0-40d0-a1d5-62acbdb7f3c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:29.217896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999c856-deb0-40d0-a1d5-62acbdb7f3c2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:29.227786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 339/678] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:29.227786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca8b5e-11a2-457f-b51a-923e1ba39c0c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:29.233947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 340/679] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:29.234729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999c856-deb0-40d0-a1d5-62acbdb7f3c2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:29.362754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999c856-deb0-40d0-a1d5-62acbdb7f3c2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:29.366254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca8b5e-11a2-457f-b51a-923e1ba39c0c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:29.406040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63377661-fc36-4bbc-98c4-533eff2e1144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:29.406675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63377661-fc36-4bbc-98c4-533eff2e1144 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.406964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63377661-fc36-4bbc-98c4-533eff2e1144 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.407245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63377661-fc36-4bbc-98c4-533eff2e1144 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.410154 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:29.415046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd3716-4339-4cbc-b108-7d9c233e7860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:29.417127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3716-4339-4cbc-b108-7d9c233e7860 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.417479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63377661-fc36-4bbc-98c4-533eff2e1144 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:29.418322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3716-4339-4cbc-b108-7d9c233e7860 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.419020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63377661-fc36-4bbc-98c4-533eff2e1144 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:29.428380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3716-4339-4cbc-b108-7d9c233e7860 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.428380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd3716-4339-4cbc-b108-7d9c233e7860 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:29.428380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd3716-4339-4cbc-b108-7d9c233e7860 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:29.428380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd3716-4339-4cbc-b108-7d9c233e7860 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-63377661-fc36-4bbc-98c4-533eff2e1144 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:29.433681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:29.435296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:29.438120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:29.438120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:29.438120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:29.438120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:29.438120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:29.438120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:29.438120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:29.438120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:29.438120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:29.438120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:29.439823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 341/680] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:29.440722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63377661-fc36-4bbc-98c4-533eff2e1144 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:29.455056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 340/681] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:29.455056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd3716-4339-4cbc-b108-7d9c233e7860 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:29.514916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:29.581014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd3716-4339-4cbc-b108-7d9c233e7860 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:29.581998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63377661-fc36-4bbc-98c4-533eff2e1144 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:29.618016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16766e75-b708-4d72-9db2-18b409757d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:29.618695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16766e75-b708-4d72-9db2-18b409757d5a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.619031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16766e75-b708-4d72-9db2-18b409757d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.619308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16766e75-b708-4d72-9db2-18b409757d5a None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.620046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16766e75-b708-4d72-9db2-18b409757d5a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:29.620353 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16766e75-b708-4d72-9db2-18b409757d5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:29.621818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16766e75-b708-4d72-9db2-18b409757d5a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:29.632624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790daad-b8ec-4664-874e-8c0c84421baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:29.633928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790daad-b8ec-4664-874e-8c0c84421baa None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.635002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790daad-b8ec-4664-874e-8c0c84421baa None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.635482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790daad-b8ec-4664-874e-8c0c84421baa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.636426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790daad-b8ec-4664-874e-8c0c84421baa None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=727f7207d82640a3a228dfb83814148b) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:29.636940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790daad-b8ec-4664-874e-8c0c84421baa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:29.639241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790daad-b8ec-4664-874e-8c0c84421baa None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:29.641824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 341/682] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Sep 17 03:30:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:29.646739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 342/683] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:29 2022] GET /identity/v3/role_assignments?user.id=727f7207d82640a3a228dfb83814148b&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:29.646950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16766e75-b708-4d72-9db2-18b409757d5a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:29.648064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790daad-b8ec-4664-874e-8c0c84421baa None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:29.781494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:29.851800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:29.857893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16766e75-b708-4d72-9db2-18b409757d5a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:29.870988 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790daad-b8ec-4664-874e-8c0c84421baa None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:29.909255 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72093e73-c5ab-432c-94cf-aafb66c03e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:29.909884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72093e73-c5ab-432c-94cf-aafb66c03e30 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.910165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72093e73-c5ab-432c-94cf-aafb66c03e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.910403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e6898-5bb6-41f0-8d05-b972b1a8ff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:29.910525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72093e73-c5ab-432c-94cf-aafb66c03e30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.911015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e6898-5bb6-41f0-8d05-b972b1a8ff57 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.911294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e6898-5bb6-41f0-8d05-b972b1a8ff57 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.911565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e6898-5bb6-41f0-8d05-b972b1a8ff57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.912293 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e6898-5bb6-41f0-8d05-b972b1a8ff57 None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=6336f0c469b84db78a90cf8435c7dbe4) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:29.912603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e6898-5bb6-41f0-8d05-b972b1a8ff57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:29.915384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e6898-5bb6-41f0-8d05-b972b1a8ff57 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:29.923341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 342/684] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:29 2022] GET /identity/v3/role_assignments?user.id=6336f0c469b84db78a90cf8435c7dbe4&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:29.937334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9f23f-21b5-4e93-9aaf-903f2ef4c113 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.937764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9f23f-21b5-4e93-9aaf-903f2ef4c113 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.938034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9f23f-21b5-4e93-9aaf-903f2ef4c113 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.939601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 343/685] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:30:29.945359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688428cf-2f32-45dc-99f3-5cf40cb8902f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:29.945638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688428cf-2f32-45dc-99f3-5cf40cb8902f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:29.945943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688428cf-2f32-45dc-99f3-5cf40cb8902f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:29.977292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-688428cf-2f32-45dc-99f3-5cf40cb8902f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:29.979740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-688428cf-2f32-45dc-99f3-5cf40cb8902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:30.000844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72093e73-c5ab-432c-94cf-aafb66c03e30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:30.002434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72093e73-c5ab-432c-94cf-aafb66c03e30 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:30.082040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:30.092409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 344/686] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:30:30.140544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 343/687] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:30.145883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688428cf-2f32-45dc-99f3-5cf40cb8902f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:30.242280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:30.251109 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688428cf-2f32-45dc-99f3-5cf40cb8902f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:30.286848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcad40-bd1f-4db8-968c-9baaf58a59c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:30.287481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcad40-bd1f-4db8-968c-9baaf58a59c1 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:30.287759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcad40-bd1f-4db8-968c-9baaf58a59c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:30.288034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcad40-bd1f-4db8-968c-9baaf58a59c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:30.299279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcad40-bd1f-4db8-968c-9baaf58a59c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:30.300750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcad40-bd1f-4db8-968c-9baaf58a59c1 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-bcdcad40-bd1f-4db8-968c-9baaf58a59c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:30.316128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:30.317112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:30.318064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:30.318064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:30.318064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:30.318064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 345/688] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:30.324077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72093e73-c5ab-432c-94cf-aafb66c03e30 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:30.389859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcad40-bd1f-4db8-968c-9baaf58a59c1 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:30.490378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72093e73-c5ab-432c-94cf-aafb66c03e30 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:30.533884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcad40-bd1f-4db8-968c-9baaf58a59c1 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:30.551463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cefce-0174-4cde-a5d9-9d1c86b2ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:30.552097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cefce-0174-4cde-a5d9-9d1c86b2ea20 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:30.552383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cefce-0174-4cde-a5d9-9d1c86b2ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:30.552660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cefce-0174-4cde-a5d9-9d1c86b2ea20 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:30.553328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cefce-0174-4cde-a5d9-9d1c86b2ea20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:30.553636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cefce-0174-4cde-a5d9-9d1c86b2ea20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:30.555092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cefce-0174-4cde-a5d9-9d1c86b2ea20 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:30.572793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e52b09-b9fe-42bc-8379-abd28f13cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:30.573451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e52b09-b9fe-42bc-8379-abd28f13cd45 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:30.574029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e52b09-b9fe-42bc-8379-abd28f13cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:30.574303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e52b09-b9fe-42bc-8379-abd28f13cd45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:30.579861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 344/689] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:30.588770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e52b09-b9fe-42bc-8379-abd28f13cd45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:30.590798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cefce-0174-4cde-a5d9-9d1c86b2ea20 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-b3e52b09-b9fe-42bc-8379-abd28f13cd45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:30.594627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:30.595577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:30.596434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 346/690] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:30.603683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e52b09-b9fe-42bc-8379-abd28f13cd45 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:30.607308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:30.654493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:30.740789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e52b09-b9fe-42bc-8379-abd28f13cd45 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:30.747780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cefce-0174-4cde-a5d9-9d1c86b2ea20 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:30.784710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a565208-66a5-48f6-9331-70a98333bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:30.785688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a565208-66a5-48f6-9331-70a98333bdca None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:30.786229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a565208-66a5-48f6-9331-70a98333bdca None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:30.786412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1096b9e-83d9-4a78-b27b-5fcc5d5a32bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:30.786905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a565208-66a5-48f6-9331-70a98333bdca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:30.787026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1096b9e-83d9-4a78-b27b-5fcc5d5a32bc None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:30.787291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1096b9e-83d9-4a78-b27b-5fcc5d5a32bc None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:30.787563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1096b9e-83d9-4a78-b27b-5fcc5d5a32bc None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:30.788336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1096b9e-83d9-4a78-b27b-5fcc5d5a32bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:30.788716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1096b9e-83d9-4a78-b27b-5fcc5d5a32bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:30.790416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1096b9e-83d9-4a78-b27b-5fcc5d5a32bc None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:30.802515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a565208-66a5-48f6-9331-70a98333bdca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:30.804201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a565208-66a5-48f6-9331-70a98333bdca None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:30.808304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 347/691] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:30.809766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dec991-d214-451a-92b6-d7fb5c6493a9 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:30.810045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dec991-d214-451a-92b6-d7fb5c6493a9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:30.810321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dec991-d214-451a-92b6-d7fb5c6493a9 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:30.813812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 348/692] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:30.814843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dec991-d214-451a-92b6-d7fb5c6493a9 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-9a565208-66a5-48f6-9331-70a98333bdca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:30.824737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:30.825683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:30.826660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:30.826660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:30.826660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:30.826660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:30.826660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:30.826660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:30.826660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:30.826660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:30.827099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 345/693] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:30.833145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9c1e1-ac0f-4ce5-bbb0-c8bb174b5327 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:30.833411 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9c1e1-ac0f-4ce5-bbb0-c8bb174b5327 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:30.833799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9c1e1-ac0f-4ce5-bbb0-c8bb174b5327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:30.862344 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-04f9c1e1-ac0f-4ce5-bbb0-c8bb174b5327 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:30.863647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-04f9c1e1-ac0f-4ce5-bbb0-c8bb174b5327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:30.901666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:30.939768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dec991-d214-451a-92b6-d7fb5c6493a9 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:30.948180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 346/694] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:30.949125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9c1e1-ac0f-4ce5-bbb0-c8bb174b5327 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:30.967870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:30.982980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ed1bb-d075-45e7-9035-6f61b731d8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:30.985852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ed1bb-d075-45e7-9035-6f61b731d8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:30.987083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ed1bb-d075-45e7-9035-6f61b731d8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:30.987988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ed1bb-d075-45e7-9035-6f61b731d8a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:31.010743 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ed1bb-d075-45e7-9035-6f61b731d8a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:31.013194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ed1bb-d075-45e7-9035-6f61b731d8a9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-2e3ed1bb-d075-45e7-9035-6f61b731d8a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:31.041867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:31.042927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:31.043887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:31.043887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:31.043887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:31.050231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 349/695] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:31.053155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ed1bb-d075-45e7-9035-6f61b731d8a9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:31.180265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ed1bb-d075-45e7-9035-6f61b731d8a9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:31.211381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:31.227342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8e08a-c2b3-436c-920e-d68fd5a1d456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:31.227927 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9c1e1-ac0f-4ce5-bbb0-c8bb174b5327 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:31.229710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8e08a-c2b3-436c-920e-d68fd5a1d456 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:31.230486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8e08a-c2b3-436c-920e-d68fd5a1d456 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:31.230957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8e08a-c2b3-436c-920e-d68fd5a1d456 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:31.232129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8e08a-c2b3-436c-920e-d68fd5a1d456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:31.232869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8e08a-c2b3-436c-920e-d68fd5a1d456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:31.235916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8e08a-c2b3-436c-920e-d68fd5a1d456 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:31.249453 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 350/696] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:31.255273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8e08a-c2b3-436c-920e-d68fd5a1d456 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:31.285368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51cf5-bc62-43f5-bfbe-5086e1c68eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:31.286167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51cf5-bc62-43f5-bfbe-5086e1c68eda None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:31.286455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51cf5-bc62-43f5-bfbe-5086e1c68eda None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:31.286778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51cf5-bc62-43f5-bfbe-5086e1c68eda None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:31.287480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51cf5-bc62-43f5-bfbe-5086e1c68eda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:31.287782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51cf5-bc62-43f5-bfbe-5086e1c68eda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:31.289460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51cf5-bc62-43f5-bfbe-5086e1c68eda None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:31.304038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 347/697] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:31.305099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51cf5-bc62-43f5-bfbe-5086e1c68eda None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:31.366444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8e08a-c2b3-436c-920e-d68fd5a1d456 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:31.381002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:31.416377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578df75b-1456-4b0d-a03f-ac15fd6cf235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:31.417340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578df75b-1456-4b0d-a03f-ac15fd6cf235 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:31.417624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578df75b-1456-4b0d-a03f-ac15fd6cf235 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:31.417932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578df75b-1456-4b0d-a03f-ac15fd6cf235 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:31.419070 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51cf5-bc62-43f5-bfbe-5086e1c68eda None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:31.430824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578df75b-1456-4b0d-a03f-ac15fd6cf235 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:31.433787 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578df75b-1456-4b0d-a03f-ac15fd6cf235 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-578df75b-1456-4b0d-a03f-ac15fd6cf235 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:31.450892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:31.451928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:31.452865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:31.452865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:31.452865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:31.452865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:31.452865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:31.452865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:31.452865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:31.452865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:31.452865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:31.452865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:31.453402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 351/698] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Sep 17 03:30:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:31.455220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578df75b-1456-4b0d-a03f-ac15fd6cf235 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:31.473777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d7ea-df0c-40aa-a5aa-b302379d1b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:31.473777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d7ea-df0c-40aa-a5aa-b302379d1b90 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:31.473777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d7ea-df0c-40aa-a5aa-b302379d1b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:31.473777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d7ea-df0c-40aa-a5aa-b302379d1b90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:31.482669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7d7ea-df0c-40aa-a5aa-b302379d1b90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:31.484128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7d7ea-df0c-40aa-a5aa-b302379d1b90 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-02c7d7ea-df0c-40aa-a5aa-b302379d1b90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:31.500371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:31.501408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:31.502421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:31.502421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:31.502421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:31.502609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:31.506178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 348/699] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:31.506663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d7ea-df0c-40aa-a5aa-b302379d1b90 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:31.621245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578df75b-1456-4b0d-a03f-ac15fd6cf235 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:31.666692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d7ea-df0c-40aa-a5aa-b302379d1b90 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:31.678094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116dd5b-eac7-4fcb-a246-713e5f8b294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:31.678702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116dd5b-eac7-4fcb-a246-713e5f8b294d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:31.678981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116dd5b-eac7-4fcb-a246-713e5f8b294d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:31.679249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116dd5b-eac7-4fcb-a246-713e5f8b294d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:31.680080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116dd5b-eac7-4fcb-a246-713e5f8b294d None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=c1136770f79a47f6a2af0b0a32259209) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:31.680387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116dd5b-eac7-4fcb-a246-713e5f8b294d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:31.682503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116dd5b-eac7-4fcb-a246-713e5f8b294d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:31.686811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:31.690136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 352/700] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:31 2022] GET /identity/v3/role_assignments?user.id=c1136770f79a47f6a2af0b0a32259209&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:31.691196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116dd5b-eac7-4fcb-a246-713e5f8b294d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:31.702311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e26e67-e302-4ea3-8b3d-ce29a8bb09e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:31.703380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e26e67-e302-4ea3-8b3d-ce29a8bb09e6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:31.703380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e26e67-e302-4ea3-8b3d-ce29a8bb09e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:31.703570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e26e67-e302-4ea3-8b3d-ce29a8bb09e6 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:31.704204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e26e67-e302-4ea3-8b3d-ce29a8bb09e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:31.704519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e26e67-e302-4ea3-8b3d-ce29a8bb09e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:31.706093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e26e67-e302-4ea3-8b3d-ce29a8bb09e6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:31.726981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 349/701] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Sep 17 03:30:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:31.732764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e26e67-e302-4ea3-8b3d-ce29a8bb09e6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:31.820814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116dd5b-eac7-4fcb-a246-713e5f8b294d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:31.826092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:31.864607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2ab57-406b-4892-bc4d-e5d37635bbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:31.865346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2ab57-406b-4892-bc4d-e5d37635bbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:31.865629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2ab57-406b-4892-bc4d-e5d37635bbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:31.865950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2ab57-406b-4892-bc4d-e5d37635bbb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:31.866605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2ab57-406b-4892-bc4d-e5d37635bbb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:31.866909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2ab57-406b-4892-bc4d-e5d37635bbb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:31.868310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2ab57-406b-4892-bc4d-e5d37635bbb2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:31.889245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e26e67-e302-4ea3-8b3d-ce29a8bb09e6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:31.889500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 353/702] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:31.890869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2ab57-406b-4892-bc4d-e5d37635bbb2 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:31.952171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c069b-b5c8-4836-a993-88976886f64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:31.952796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c069b-b5c8-4836-a993-88976886f64b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:31.953080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c069b-b5c8-4836-a993-88976886f64b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:31.953356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c069b-b5c8-4836-a993-88976886f64b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:32.005408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2ab57-406b-4892-bc4d-e5d37635bbb2 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:32.018192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:32.034831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c069b-b5c8-4836-a993-88976886f64b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:32.036531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c069b-b5c8-4836-a993-88976886f64b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:32.045039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1805c-717e-45d1-b747-812af7e76200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:32.045654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1805c-717e-45d1-b747-812af7e76200 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:32.046034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1805c-717e-45d1-b747-812af7e76200 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:32.046310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1805c-717e-45d1-b747-812af7e76200 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:32.047189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1805c-717e-45d1-b747-812af7e76200 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:32.048995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1805c-717e-45d1-b747-812af7e76200 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:32.162252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 350/704] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:32.162445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 354/704] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:30:31 2022] POST /identity/v3/users => generated 254 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:30:32.168580 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1805c-717e-45d1-b747-812af7e76200 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:32.198967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:32.305519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1805c-717e-45d1-b747-812af7e76200 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:32.362472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f463c14-c84d-41fb-88ff-23f201b81923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:32.363128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f463c14-c84d-41fb-88ff-23f201b81923 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:32.363407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f463c14-c84d-41fb-88ff-23f201b81923 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:32.363677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f463c14-c84d-41fb-88ff-23f201b81923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:32.375643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f463c14-c84d-41fb-88ff-23f201b81923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:32.377116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f463c14-c84d-41fb-88ff-23f201b81923 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-9f463c14-c84d-41fb-88ff-23f201b81923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:32.393702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:32.394745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:32.395660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:32.395660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:32.395660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:32.395660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 355/705] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:32.400924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c069b-b5c8-4836-a993-88976886f64b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:32.575596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed235fc-011d-4491-b76f-0580b7e054f4 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:32.575788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed235fc-011d-4491-b76f-0580b7e054f4 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:32.576068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed235fc-011d-4491-b76f-0580b7e054f4 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:32.577665 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 356/706] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:30:32.582811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee73542-3925-47d4-a2e1-d97a815bbbb0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:32.583099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee73542-3925-47d4-a2e1-d97a815bbbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:32.583372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee73542-3925-47d4-a2e1-d97a815bbbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:32.605155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-0ee73542-3925-47d4-a2e1-d97a815bbbb0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:32.606410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0ee73542-3925-47d4-a2e1-d97a815bbbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:32.609454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:32.638338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c069b-b5c8-4836-a993-88976886f64b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:32.685786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:32.693622 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 357/707] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:30:32.697754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70f97f-1658-4554-b03e-081a004b35ed None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:32.697754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70f97f-1658-4554-b03e-081a004b35ed None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:32.697754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70f97f-1658-4554-b03e-081a004b35ed None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:32.698048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b757028-87ea-4240-a94e-94cd2c36dcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:32.698467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b757028-87ea-4240-a94e-94cd2c36dcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:32.700182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b757028-87ea-4240-a94e-94cd2c36dcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:32.700182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b757028-87ea-4240-a94e-94cd2c36dcd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:32.700182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b757028-87ea-4240-a94e-94cd2c36dcd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:32.700435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b757028-87ea-4240-a94e-94cd2c36dcd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:32.701013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 358/708] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:32.701867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b757028-87ea-4240-a94e-94cd2c36dcd0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:32.703578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ef417-3da7-4d95-be7a-2aef7564ca40 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:32.703855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ef417-3da7-4d95-be7a-2aef7564ca40 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:32.704241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ef417-3da7-4d95-be7a-2aef7564ca40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:32.726281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 351/709] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:32.733201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b757028-87ea-4240-a94e-94cd2c36dcd0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:32.739819 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ea2ef417-3da7-4d95-be7a-2aef7564ca40 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:32.741020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ea2ef417-3da7-4d95-be7a-2aef7564ca40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:32.834381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 359/710] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:32.930113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b757028-87ea-4240-a94e-94cd2c36dcd0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:32.986066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76cc4b-2ee8-4d31-a6ba-8b3d16c7c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:32.986701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76cc4b-2ee8-4d31-a6ba-8b3d16c7c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:32.986980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76cc4b-2ee8-4d31-a6ba-8b3d16c7c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:32.987254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76cc4b-2ee8-4d31-a6ba-8b3d16c7c0b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:32.995007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76cc4b-2ee8-4d31-a6ba-8b3d16c7c0b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:32.996504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76cc4b-2ee8-4d31-a6ba-8b3d16c7c0b8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:32.998305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-6e76cc4b-2ee8-4d31-a6ba-8b3d16c7c0b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:33.010669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:33.013163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:33.014321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:33.014321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:33.014321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:33.014321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:33.014321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:33.014321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:33.014321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:33.014321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:33.014321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 352/711] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:33.017546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ef417-3da7-4d95-be7a-2aef7564ca40 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:33.111130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76cc4b-2ee8-4d31-a6ba-8b3d16c7c0b8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:33.147342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ef417-3da7-4d95-be7a-2aef7564ca40 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:33.180224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:33.192938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184d6fd-7bd4-4c24-a639-4b3604a59abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:33.193877 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184d6fd-7bd4-4c24-a639-4b3604a59abe None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:33.194326 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184d6fd-7bd4-4c24-a639-4b3604a59abe None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:33.194765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184d6fd-7bd4-4c24-a639-4b3604a59abe None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:33.198876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184d6fd-7bd4-4c24-a639-4b3604a59abe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:33.199402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184d6fd-7bd4-4c24-a639-4b3604a59abe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:33.202727 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184d6fd-7bd4-4c24-a639-4b3604a59abe None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:33.223019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 360/712] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:33.223971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184d6fd-7bd4-4c24-a639-4b3604a59abe None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:33.268404 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76cc4b-2ee8-4d31-a6ba-8b3d16c7c0b8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:33.304733 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58209b51-1c14-40f1-8a00-c4de2bf22cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:33.305828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58209b51-1c14-40f1-8a00-c4de2bf22cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:33.306934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58209b51-1c14-40f1-8a00-c4de2bf22cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:33.307406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58209b51-1c14-40f1-8a00-c4de2bf22cd0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:33.308238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58209b51-1c14-40f1-8a00-c4de2bf22cd0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:33.309847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58209b51-1c14-40f1-8a00-c4de2bf22cd0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-58209b51-1c14-40f1-8a00-c4de2bf22cd0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:33.321039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:33.322172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 03:30:33.323211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:33.326210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 353/713] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:33 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:33.331893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58209b51-1c14-40f1-8a00-c4de2bf22cd0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:33.334659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184d6fd-7bd4-4c24-a639-4b3604a59abe None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:33.370844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ad3a0-61f8-4451-8e6b-196b9a895e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:33.371145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ad3a0-61f8-4451-8e6b-196b9a895e35 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:33.371383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ad3a0-61f8-4451-8e6b-196b9a895e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:33.371655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ad3a0-61f8-4451-8e6b-196b9a895e35 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:33.372232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ad3a0-61f8-4451-8e6b-196b9a895e35 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:33.373598 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ad3a0-61f8-4451-8e6b-196b9a895e35 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-a73ad3a0-61f8-4451-8e6b-196b9a895e35 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:33.385333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:33.386391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:33.387231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 361/714] 10.222.0.42 () {56 vars in 1129 bytes} [Sat Sep 17 03:30:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:33.388049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ad3a0-61f8-4451-8e6b-196b9a895e35 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:33.403765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:33.447379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:33.452097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58209b51-1c14-40f1-8a00-c4de2bf22cd0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:33.503716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843a199-37d8-46fa-9999-d94c64c1d404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:33.504459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843a199-37d8-46fa-9999-d94c64c1d404 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:33.504804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843a199-37d8-46fa-9999-d94c64c1d404 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:33.505151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843a199-37d8-46fa-9999-d94c64c1d404 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:33.505990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843a199-37d8-46fa-9999-d94c64c1d404 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=9901608b62344115aa561d00bfe88c1c) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:33.506370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843a199-37d8-46fa-9999-d94c64c1d404 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:33.508636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ad3a0-61f8-4451-8e6b-196b9a895e35 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:33.511259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843a199-37d8-46fa-9999-d94c64c1d404 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:33.530204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 354/715] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:33 2022] GET /identity/v3/role_assignments?user.id=9901608b62344115aa561d00bfe88c1c&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:33.531302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843a199-37d8-46fa-9999-d94c64c1d404 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:33.555932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353feb6-49aa-440a-aca1-ea97856b6c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:33.557340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353feb6-49aa-440a-aca1-ea97856b6c56 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:33.560285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353feb6-49aa-440a-aca1-ea97856b6c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:33.560573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353feb6-49aa-440a-aca1-ea97856b6c56 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:33.561262 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353feb6-49aa-440a-aca1-ea97856b6c56 None admin] RBAC: Adding query filter params (name=network) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:33.563886 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353feb6-49aa-440a-aca1-ea97856b6c56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:33.565377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353feb6-49aa-440a-aca1-ea97856b6c56 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:33.578265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 362/716] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:30:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:33.579655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353feb6-49aa-440a-aca1-ea97856b6c56 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:33.723607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843a199-37d8-46fa-9999-d94c64c1d404 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:33.755535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353feb6-49aa-440a-aca1-ea97856b6c56 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:33.756132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:33.770096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c182003-e5e1-4f1e-9884-6abfd8bfa1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:33.770716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c182003-e5e1-4f1e-9884-6abfd8bfa1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:33.770996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c182003-e5e1-4f1e-9884-6abfd8bfa1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:33.771263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c182003-e5e1-4f1e-9884-6abfd8bfa1d4 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:33.771891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c182003-e5e1-4f1e-9884-6abfd8bfa1d4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:33.772191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c182003-e5e1-4f1e-9884-6abfd8bfa1d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:33.775519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c182003-e5e1-4f1e-9884-6abfd8bfa1d4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:33.784459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 355/717] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Sep 17 03:30:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:33.788641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c182003-e5e1-4f1e-9884-6abfd8bfa1d4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:33.791808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:33.847766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b864d-b22b-4251-97a5-024f5c587c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:33.848396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b864d-b22b-4251-97a5-024f5c587c42 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:33.848673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b864d-b22b-4251-97a5-024f5c587c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:33.849010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b864d-b22b-4251-97a5-024f5c587c42 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:33.849581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b864d-b22b-4251-97a5-024f5c587c42 None admin] RBAC: Adding query filter params (type=network) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:33.849911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b864d-b22b-4251-97a5-024f5c587c42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:33.851304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b864d-b22b-4251-97a5-024f5c587c42 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:33.869609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 363/718] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:30:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:33.948868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c182003-e5e1-4f1e-9884-6abfd8bfa1d4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:34.005549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae77b9b-7404-49c3-a49b-c3ae9fbc6816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:34.006570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae77b9b-7404-49c3-a49b-c3ae9fbc6816 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:34.007093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae77b9b-7404-49c3-a49b-c3ae9fbc6816 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:34.007584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae77b9b-7404-49c3-a49b-c3ae9fbc6816 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:34.008460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae77b9b-7404-49c3-a49b-c3ae9fbc6816 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:34.008986 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae77b9b-7404-49c3-a49b-c3ae9fbc6816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:34.010902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae77b9b-7404-49c3-a49b-c3ae9fbc6816 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:34.025357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 356/719] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Sep 17 03:30:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:34.521941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565043f2-23e8-41be-b5b0-59d42fd03ebf None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:34.522489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565043f2-23e8-41be-b5b0-59d42fd03ebf None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:34.522489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565043f2-23e8-41be-b5b0-59d42fd03ebf None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:34.524193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 364/720] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:34.530560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b85c7-4b16-4e31-84fe-97cbd4775c69 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:34.530913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b85c7-4b16-4e31-84fe-97cbd4775c69 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:34.531198 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b85c7-4b16-4e31-84fe-97cbd4775c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:34.554224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-800b85c7-4b16-4e31-84fe-97cbd4775c69 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:34.555430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-800b85c7-4b16-4e31-84fe-97cbd4775c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:34.614423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:34.632105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 357/721] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:34.905655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565043f2-23e8-41be-b5b0-59d42fd03ebf None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:35.034252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565043f2-23e8-41be-b5b0-59d42fd03ebf None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:35.070359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccecfa-6d9e-4c17-9f1d-ea336006e16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:35.070913 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccecfa-6d9e-4c17-9f1d-ea336006e16b None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:35.071257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccecfa-6d9e-4c17-9f1d-ea336006e16b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:35.071539 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccecfa-6d9e-4c17-9f1d-ea336006e16b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:35.082743 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccecfa-6d9e-4c17-9f1d-ea336006e16b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-2bccecfa-6d9e-4c17-9f1d-ea336006e16b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:35.088555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:35.089507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:35.090384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:35.094082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 365/722] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:35.095856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b85c7-4b16-4e31-84fe-97cbd4775c69 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:35.172753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb40248-f217-4325-8bde-7ea69f949582 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:35.173074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb40248-f217-4325-8bde-7ea69f949582 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:35.173280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb40248-f217-4325-8bde-7ea69f949582 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:35.174904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 366/723] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:30:35.179718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d6f1c-0c40-4c2d-b3f4-572764ebf259 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:35.179994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d6f1c-0c40-4c2d-b3f4-572764ebf259 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:35.180264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d6f1c-0c40-4c2d-b3f4-572764ebf259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:35.207163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-980d6f1c-0c40-4c2d-b3f4-572764ebf259 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:35.208442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-980d6f1c-0c40-4c2d-b3f4-572764ebf259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:35.238906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b85c7-4b16-4e31-84fe-97cbd4775c69 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:35.262519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:35.296299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 367/724] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 03:30:35.324777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111609e-9901-413e-b265-691f7c3f0a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:35.325322 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111609e-9901-413e-b265-691f7c3f0a06 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:35.325605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111609e-9901-413e-b265-691f7c3f0a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:35.325913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111609e-9901-413e-b265-691f7c3f0a06 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:35.326591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111609e-9901-413e-b265-691f7c3f0a06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:35.326896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111609e-9901-413e-b265-691f7c3f0a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:35.328271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111609e-9901-413e-b265-691f7c3f0a06 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:35.341781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 358/725] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:35.347087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d6f1c-0c40-4c2d-b3f4-572764ebf259 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:35.487666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d6f1c-0c40-4c2d-b3f4-572764ebf259 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:35.534630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ccfea-608b-4046-b61a-24bf308266ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:35.536113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ccfea-608b-4046-b61a-24bf308266ba None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:35.536697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ccfea-608b-4046-b61a-24bf308266ba None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:35.537170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ccfea-608b-4046-b61a-24bf308266ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:35.551590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ccfea-608b-4046-b61a-24bf308266ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:35.553251 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ccfea-608b-4046-b61a-24bf308266ba None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-576ccfea-608b-4046-b61a-24bf308266ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:35.577018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:35.578073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:35.578997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:35.578997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:35.578997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:35.578997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 368/726] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:35.583798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111609e-9901-413e-b265-691f7c3f0a06 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:35.623748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ccfea-608b-4046-b61a-24bf308266ba None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:35.653856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:35.689095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:35.785670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ccfea-608b-4046-b61a-24bf308266ba None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:35.790596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111609e-9901-413e-b265-691f7c3f0a06 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:35.848077 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba429ae1-1923-4fee-88e8-9e23d33fc5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:35.848623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba429ae1-1923-4fee-88e8-9e23d33fc5ed None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:35.848896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba429ae1-1923-4fee-88e8-9e23d33fc5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:35.849165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba429ae1-1923-4fee-88e8-9e23d33fc5ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:35.850234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634051c-3622-44d5-be08-9de738e12b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:35.850823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634051c-3622-44d5-be08-9de738e12b47 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:35.851093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634051c-3622-44d5-be08-9de738e12b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:35.851359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634051c-3622-44d5-be08-9de738e12b47 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:35.851991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634051c-3622-44d5-be08-9de738e12b47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:35.852286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634051c-3622-44d5-be08-9de738e12b47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:35.853749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634051c-3622-44d5-be08-9de738e12b47 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:35.874431 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba429ae1-1923-4fee-88e8-9e23d33fc5ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:35.879311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 359/727] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:35.880180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634051c-3622-44d5-be08-9de738e12b47 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-ba429ae1-1923-4fee-88e8-9e23d33fc5ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:35.881221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:35.882206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:35.883067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:35.890407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 369/728] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:35.895994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba429ae1-1923-4fee-88e8-9e23d33fc5ed None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:36.016986 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634051c-3622-44d5-be08-9de738e12b47 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:36.032474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:36.054141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113cd5a6-2ce7-4f97-82a9-e8f9259629c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:36.054940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cd5a6-2ce7-4f97-82a9-e8f9259629c9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.055232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cd5a6-2ce7-4f97-82a9-e8f9259629c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.055517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cd5a6-2ce7-4f97-82a9-e8f9259629c9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.064288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cd5a6-2ce7-4f97-82a9-e8f9259629c9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:36.065773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cd5a6-2ce7-4f97-82a9-e8f9259629c9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-113cd5a6-2ce7-4f97-82a9-e8f9259629c9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:36.080320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:36.081377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:36.082408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 360/729] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:36.085773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30bca1-09b4-4b76-a7c5-d3aedacba7aa None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.085773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30bca1-09b4-4b76-a7c5-d3aedacba7aa None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.085773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30bca1-09b4-4b76-a7c5-d3aedacba7aa None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.089871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 361/730] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:36.094732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30bca1-09b4-4b76-a7c5-d3aedacba7aa None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:36.105489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:36.124676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba429ae1-1923-4fee-88e8-9e23d33fc5ed None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:36.169073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309d705-8b16-42bd-8b8f-40d22dc5afe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:36.170160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309d705-8b16-42bd-8b8f-40d22dc5afe9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.170924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309d705-8b16-42bd-8b8f-40d22dc5afe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.171633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309d705-8b16-42bd-8b8f-40d22dc5afe9 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.172468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309d705-8b16-42bd-8b8f-40d22dc5afe9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:36.173082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309d705-8b16-42bd-8b8f-40d22dc5afe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:36.175280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309d705-8b16-42bd-8b8f-40d22dc5afe9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:36.191573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 370/731] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:36.191573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd64f7a-4afa-4843-a635-854c6cfbf324 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.191573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd64f7a-4afa-4843-a635-854c6cfbf324 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.191874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd64f7a-4afa-4843-a635-854c6cfbf324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.211591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-7bd64f7a-4afa-4843-a635-854c6cfbf324 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:36.212756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7bd64f7a-4afa-4843-a635-854c6cfbf324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:36.222266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30bca1-09b4-4b76-a7c5-d3aedacba7aa None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:36.260480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2d79b-ddc2-4b0c-8a52-a1c810828e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:36.261447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2d79b-ddc2-4b0c-8a52-a1c810828e8c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.262000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2d79b-ddc2-4b0c-8a52-a1c810828e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.262494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2d79b-ddc2-4b0c-8a52-a1c810828e8c None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.263422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2d79b-ddc2-4b0c-8a52-a1c810828e8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:36.263920 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2d79b-ddc2-4b0c-8a52-a1c810828e8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:36.265797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2d79b-ddc2-4b0c-8a52-a1c810828e8c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:36.285534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 362/732] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Sep 17 03:30:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:36.286758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2d79b-ddc2-4b0c-8a52-a1c810828e8c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:36.289863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 371/733] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:36.291854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd64f7a-4afa-4843-a635-854c6cfbf324 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:36.340254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:36.426146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:36.460971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd64f7a-4afa-4843-a635-854c6cfbf324 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:36.475942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2d79b-ddc2-4b0c-8a52-a1c810828e8c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:36.514980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d9a6c-241b-4ff2-99af-6522215b5c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:36.515956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d9a6c-241b-4ff2-99af-6522215b5c7e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.516568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d9a6c-241b-4ff2-99af-6522215b5c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.517049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d9a6c-241b-4ff2-99af-6522215b5c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.528855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d9a6c-241b-4ff2-99af-6522215b5c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:36.530611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d9a6c-241b-4ff2-99af-6522215b5c7e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d62d9a6c-241b-4ff2-99af-6522215b5c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:36.546892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:36.547933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:36.548844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:36.548844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:36.548844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:36.551305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 363/734] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:36.557639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501f515-cf2f-4384-b557-a17c164d0606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:36.557639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501f515-cf2f-4384-b557-a17c164d0606 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.557639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501f515-cf2f-4384-b557-a17c164d0606 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.557639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501f515-cf2f-4384-b557-a17c164d0606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.565308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d9a6c-241b-4ff2-99af-6522215b5c7e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:36.646003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501f515-cf2f-4384-b557-a17c164d0606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:36.647466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501f515-cf2f-4384-b557-a17c164d0606 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:36.668423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:36.699498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d9a6c-241b-4ff2-99af-6522215b5c7e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:36.736080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:36.741413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0863c2-7d2d-4592-9f47-dd5da54d1f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:36.742294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0863c2-7d2d-4592-9f47-dd5da54d1f81 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.742727 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0863c2-7d2d-4592-9f47-dd5da54d1f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.743147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0863c2-7d2d-4592-9f47-dd5da54d1f81 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.744124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0863c2-7d2d-4592-9f47-dd5da54d1f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:36.744522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0863c2-7d2d-4592-9f47-dd5da54d1f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:36.745930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0863c2-7d2d-4592-9f47-dd5da54d1f81 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:36.764159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 372/735] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 03:30:36.764159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501f515-cf2f-4384-b557-a17c164d0606 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:36.765253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 364/736] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:36.767379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f09dc-4cb1-4139-8863-644009f9206b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.767667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f09dc-4cb1-4139-8863-644009f9206b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.767942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f09dc-4cb1-4139-8863-644009f9206b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.773261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 365/737] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:36.773261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f09dc-4cb1-4139-8863-644009f9206b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:36.865530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501f515-cf2f-4384-b557-a17c164d0606 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:36.887095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f09dc-4cb1-4139-8863-644009f9206b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:36.899203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbc1e4-c185-4198-9f76-fb71578d4348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:36.899903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbc1e4-c185-4198-9f76-fb71578d4348 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.900267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbc1e4-c185-4198-9f76-fb71578d4348 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.900537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbc1e4-c185-4198-9f76-fb71578d4348 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.910436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbc1e4-c185-4198-9f76-fb71578d4348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-ebbbc1e4-c185-4198-9f76-fb71578d4348 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:36.915178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:36.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 373/738] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:36.917714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbc1e4-c185-4198-9f76-fb71578d4348 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:36.925342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc79362-9754-4a35-9ad5-2ac8edbfcfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:36.925998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc79362-9754-4a35-9ad5-2ac8edbfcfef None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.926281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc79362-9754-4a35-9ad5-2ac8edbfcfef None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.926552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc79362-9754-4a35-9ad5-2ac8edbfcfef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.929194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:36.937950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc79362-9754-4a35-9ad5-2ac8edbfcfef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:36.939455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc79362-9754-4a35-9ad5-2ac8edbfcfef None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:36.952725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4bc79362-9754-4a35-9ad5-2ac8edbfcfef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:36.954447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:36.955440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:36.956351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:36.956351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:36.956351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:36.960790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 366/739] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:36.962503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2692e-6c56-4f35-b553-232944c03e65 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:36.962833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2692e-6c56-4f35-b553-232944c03e65 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:36.967992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2692e-6c56-4f35-b553-232944c03e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:36.981356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4bf2692e-6c56-4f35-b553-232944c03e65 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:36.982569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4bf2692e-6c56-4f35-b553-232944c03e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:37.017052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbc1e4-c185-4198-9f76-fb71578d4348 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:37.047616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 367/740] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:37.048845 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f99e2-f8e8-4bdb-9218-218c8c3a8e7b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.049131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f99e2-f8e8-4bdb-9218-218c8c3a8e7b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.049402 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f99e2-f8e8-4bdb-9218-218c8c3a8e7b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.050654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 368/741] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:37.051349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f99e2-f8e8-4bdb-9218-218c8c3a8e7b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:37.057627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f89216-cfff-49be-ada6-4755e1b6a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:37.058832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f89216-cfff-49be-ada6-4755e1b6a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.058832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f89216-cfff-49be-ada6-4755e1b6a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.058832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f89216-cfff-49be-ada6-4755e1b6a1b1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.066071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f89216-cfff-49be-ada6-4755e1b6a1b1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:37.067531 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f89216-cfff-49be-ada6-4755e1b6a1b1 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-f4f89216-cfff-49be-ada6-4755e1b6a1b1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:37.079112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:37.080067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 374/742] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Sep 17 03:30:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:37.081051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f89216-cfff-49be-ada6-4755e1b6a1b1 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:37.163809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f99e2-f8e8-4bdb-9218-218c8c3a8e7b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:37.179327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:37.182198 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:37.185065 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f89216-cfff-49be-ada6-4755e1b6a1b1 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:37.199346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43485253-d514-4bb4-a2f2-624ffdc9fab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:37.199951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43485253-d514-4bb4-a2f2-624ffdc9fab1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.200233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43485253-d514-4bb4-a2f2-624ffdc9fab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.200505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43485253-d514-4bb4-a2f2-624ffdc9fab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.201167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43485253-d514-4bb4-a2f2-624ffdc9fab1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:37.201469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43485253-d514-4bb4-a2f2-624ffdc9fab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:37.202904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43485253-d514-4bb4-a2f2-624ffdc9fab1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:37.215748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 369/743] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:37.215748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ed7c-0928-4eee-a61b-edf9019b9bd8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.215748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ed7c-0928-4eee-a61b-edf9019b9bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.215748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ed7c-0928-4eee-a61b-edf9019b9bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.226119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753b56f-ab9f-4a92-b479-c75dba0c429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:37.226723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753b56f-ab9f-4a92-b479-c75dba0c429a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.226998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753b56f-ab9f-4a92-b479-c75dba0c429a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.227269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753b56f-ab9f-4a92-b479-c75dba0c429a None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.228014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753b56f-ab9f-4a92-b479-c75dba0c429a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:37.228825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753b56f-ab9f-4a92-b479-c75dba0c429a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:37.230321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753b56f-ab9f-4a92-b479-c75dba0c429a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:37.231372 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9100ed7c-0928-4eee-a61b-edf9019b9bd8 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:37.232721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9100ed7c-0928-4eee-a61b-edf9019b9bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:37.244312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 375/744] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:37.245139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753b56f-ab9f-4a92-b479-c75dba0c429a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:37.309788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 370/745] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 03:30:37.310747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100ed7c-0928-4eee-a61b-edf9019b9bd8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:37.354506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753b56f-ab9f-4a92-b479-c75dba0c429a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:37.391563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066c5b3-90d9-4b57-98ee-3eb09d3be363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:37.392160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066c5b3-90d9-4b57-98ee-3eb09d3be363 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.392432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066c5b3-90d9-4b57-98ee-3eb09d3be363 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.392698 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066c5b3-90d9-4b57-98ee-3eb09d3be363 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.393409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066c5b3-90d9-4b57-98ee-3eb09d3be363 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:37.393708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066c5b3-90d9-4b57-98ee-3eb09d3be363 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:37.395178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066c5b3-90d9-4b57-98ee-3eb09d3be363 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:37.412744 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 376/746] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Sep 17 03:30:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:37.418215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066c5b3-90d9-4b57-98ee-3eb09d3be363 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:37.425890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100ed7c-0928-4eee-a61b-edf9019b9bd8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:37.435086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:37.438347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:37.471878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ffac-9cad-4d60-82e5-ba1d1fdba99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:37.472443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ffac-9cad-4d60-82e5-ba1d1fdba99e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.472722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ffac-9cad-4d60-82e5-ba1d1fdba99e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.472993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ffac-9cad-4d60-82e5-ba1d1fdba99e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.487303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331ffac-9cad-4d60-82e5-ba1d1fdba99e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:37.488744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331ffac-9cad-4d60-82e5-ba1d1fdba99e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3331ffac-9cad-4d60-82e5-ba1d1fdba99e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:37.504474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:37.505629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:37.506847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:37.506847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:37.506847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:37.510693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 371/747] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:37.511511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ffac-9cad-4d60-82e5-ba1d1fdba99e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:37.550813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066c5b3-90d9-4b57-98ee-3eb09d3be363 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:37.602472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c2eb3-c537-4e9b-9947-ed3829f271a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:37.603762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c2eb3-c537-4e9b-9947-ed3829f271a0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.603907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c2eb3-c537-4e9b-9947-ed3829f271a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.604174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c2eb3-c537-4e9b-9947-ed3829f271a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.612784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c2eb3-c537-4e9b-9947-ed3829f271a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:37.614305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c2eb3-c537-4e9b-9947-ed3829f271a0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-e31c2eb3-c537-4e9b-9947-ed3829f271a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:37.627120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:37.628190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:37.629099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:37.629099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:37.629099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:37.629099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:37.629099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:37.629099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:37.629099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:37.629099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:37.629099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 377/748] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:37.629099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c2eb3-c537-4e9b-9947-ed3829f271a0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:37.643651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ffac-9cad-4d60-82e5-ba1d1fdba99e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:37.682882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5bd43-302e-40ad-bd6f-07bd2d949715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:37.683554 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5bd43-302e-40ad-bd6f-07bd2d949715 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.683839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5bd43-302e-40ad-bd6f-07bd2d949715 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.684113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5bd43-302e-40ad-bd6f-07bd2d949715 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.684768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5bd43-302e-40ad-bd6f-07bd2d949715 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:37.686180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5bd43-302e-40ad-bd6f-07bd2d949715 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:37.723154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:37.751521 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c2eb3-c537-4e9b-9947-ed3829f271a0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:37.778853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 372/749] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:30:37 2022] POST /identity/v3/services => generated 240 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:30:37.779923 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5bd43-302e-40ad-bd6f-07bd2d949715 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:37.807946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0c8d8-21dd-4bfa-af99-bb9c16f7ac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:37.810459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0c8d8-21dd-4bfa-af99-bb9c16f7ac17 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.810459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0c8d8-21dd-4bfa-af99-bb9c16f7ac17 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.810459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0c8d8-21dd-4bfa-af99-bb9c16f7ac17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.827664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:37.906213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0c8d8-21dd-4bfa-af99-bb9c16f7ac17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:37.907988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0c8d8-21dd-4bfa-af99-bb9c16f7ac17 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:37.927239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5bd43-302e-40ad-bd6f-07bd2d949715 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:37.979092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e3f59-1485-4c65-9a35-51054ad71a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:37.980062 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e3f59-1485-4c65-9a35-51054ad71a8c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:37.980466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e3f59-1485-4c65-9a35-51054ad71a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:37.980748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e3f59-1485-4c65-9a35-51054ad71a8c None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:37.982045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e3f59-1485-4c65-9a35-51054ad71a8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:37.982045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e3f59-1485-4c65-9a35-51054ad71a8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:37.983113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e3f59-1485-4c65-9a35-51054ad71a8c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:38.001119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 373/750] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:38.002130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e3f59-1485-4c65-9a35-51054ad71a8c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:38.013020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:38.052674 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 378/751] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:37 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 03:30:38.054233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0c8d8-21dd-4bfa-af99-bb9c16f7ac17 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:38.123643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:38.144917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e3f59-1485-4c65-9a35-51054ad71a8c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:38.178708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0c8d8-21dd-4bfa-af99-bb9c16f7ac17 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:38.187592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12625ff-1588-4be4-b245-f49b952e20c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:38.188199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12625ff-1588-4be4-b245-f49b952e20c2 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:38.188479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12625ff-1588-4be4-b245-f49b952e20c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:38.188754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12625ff-1588-4be4-b245-f49b952e20c2 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:38.189416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12625ff-1588-4be4-b245-f49b952e20c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:38.190812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12625ff-1588-4be4-b245-f49b952e20c2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:38.222255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76230c51-798a-4686-87d5-f2f25ca19933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:38.222869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76230c51-798a-4686-87d5-f2f25ca19933 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:38.223145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76230c51-798a-4686-87d5-f2f25ca19933 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:38.223414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76230c51-798a-4686-87d5-f2f25ca19933 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:38.224098 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76230c51-798a-4686-87d5-f2f25ca19933 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:38.224402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76230c51-798a-4686-87d5-f2f25ca19933 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:38.226204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76230c51-798a-4686-87d5-f2f25ca19933 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:38.241602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 379/752] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:38.242436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76230c51-798a-4686-87d5-f2f25ca19933 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:38.266969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 374/753] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:30:38 2022] POST /identity/v3/services => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:30:38.267887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12625ff-1588-4be4-b245-f49b952e20c2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:38.286310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:38.404203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76230c51-798a-4686-87d5-f2f25ca19933 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:38.508711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45166bb9-acf1-4d4a-87bb-8d6dbcc4ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:38.510529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45166bb9-acf1-4d4a-87bb-8d6dbcc4ebbb None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:38.511012 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45166bb9-acf1-4d4a-87bb-8d6dbcc4ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:38.511524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45166bb9-acf1-4d4a-87bb-8d6dbcc4ebbb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:38.517201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12625ff-1588-4be4-b245-f49b952e20c2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:38.523646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45166bb9-acf1-4d4a-87bb-8d6dbcc4ebbb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:38.527962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45166bb9-acf1-4d4a-87bb-8d6dbcc4ebbb None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-45166bb9-acf1-4d4a-87bb-8d6dbcc4ebbb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:38.552027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:38.553067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 380/754] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Sep 17 03:30:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:38.554173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45166bb9-acf1-4d4a-87bb-8d6dbcc4ebbb None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:38.585085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f6fbe-abba-416f-92b4-299cb107d83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:38.585749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f6fbe-abba-416f-92b4-299cb107d83e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:38.586061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f6fbe-abba-416f-92b4-299cb107d83e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:38.586754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f6fbe-abba-416f-92b4-299cb107d83e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:38.596795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:38.604618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f6fbe-abba-416f-92b4-299cb107d83e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:38.606874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f6fbe-abba-416f-92b4-299cb107d83e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a52f6fbe-abba-416f-92b4-299cb107d83e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:38.625586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:38.626782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:38.627995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:38.627995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:38.627995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:38.627995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 375/755] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:38.628242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f6fbe-abba-416f-92b4-299cb107d83e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:38.688882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:38.713753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45166bb9-acf1-4d4a-87bb-8d6dbcc4ebbb None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:38.756542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a011c5-9149-47cc-acc6-6b7ca77b3f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:38.757159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a011c5-9149-47cc-acc6-6b7ca77b3f13 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:38.761813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a011c5-9149-47cc-acc6-6b7ca77b3f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:38.761813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a011c5-9149-47cc-acc6-6b7ca77b3f13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:38.761813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a011c5-9149-47cc-acc6-6b7ca77b3f13 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=c1136770f79a47f6a2af0b0a32259209) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:38.761813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a011c5-9149-47cc-acc6-6b7ca77b3f13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:38.761813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a011c5-9149-47cc-acc6-6b7ca77b3f13 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:38.769222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 381/756] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:38 2022] GET /identity/v3/role_assignments?user.id=c1136770f79a47f6a2af0b0a32259209&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:38.770155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a011c5-9149-47cc-acc6-6b7ca77b3f13 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:38.856845 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f6fbe-abba-416f-92b4-299cb107d83e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:38.916433 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a011c5-9149-47cc-acc6-6b7ca77b3f13 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:38.958566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467a900-905e-40e2-957b-b6a03a2851e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:38.959193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467a900-905e-40e2-957b-b6a03a2851e9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:38.959474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467a900-905e-40e2-957b-b6a03a2851e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:38.959752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467a900-905e-40e2-957b-b6a03a2851e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:38.960410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467a900-905e-40e2-957b-b6a03a2851e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:38.960714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467a900-905e-40e2-957b-b6a03a2851e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:38.963194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467a900-905e-40e2-957b-b6a03a2851e9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:38.977495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 382/757] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:38.986487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467a900-905e-40e2-957b-b6a03a2851e9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:38.992441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721933b-1afe-4593-97e1-b338ef6f002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:38.993073 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721933b-1afe-4593-97e1-b338ef6f002d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:38.993377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721933b-1afe-4593-97e1-b338ef6f002d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:38.993688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721933b-1afe-4593-97e1-b338ef6f002d None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:38.997750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721933b-1afe-4593-97e1-b338ef6f002d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:38.997750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721933b-1afe-4593-97e1-b338ef6f002d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:38.997750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721933b-1afe-4593-97e1-b338ef6f002d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:39.017861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:39.044508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 376/758] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Sep 17 03:30:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:39.049758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721933b-1afe-4593-97e1-b338ef6f002d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:39.157831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:39.222282 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467a900-905e-40e2-957b-b6a03a2851e9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:39.257592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721933b-1afe-4593-97e1-b338ef6f002d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:39.301585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacf5d1-176f-49a7-a542-43e232aff4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:39.303215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacf5d1-176f-49a7-a542-43e232aff4dc None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:39.303852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacf5d1-176f-49a7-a542-43e232aff4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:39.304928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacf5d1-176f-49a7-a542-43e232aff4dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:39.317017 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacf5d1-176f-49a7-a542-43e232aff4dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:39.319255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacf5d1-176f-49a7-a542-43e232aff4dc None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:39.326355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0d856-da67-417c-880b-3015c6e843e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:39.327593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0d856-da67-417c-880b-3015c6e843e6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:39.328075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0d856-da67-417c-880b-3015c6e843e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:39.328546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0d856-da67-417c-880b-3015c6e843e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-3aacf5d1-176f-49a7-a542-43e232aff4dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:39.340656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:39.341579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:39.342522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:39.342522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:39.342522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:39.342522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:39.342522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:39.342522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:39.342522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:39.342522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:39.350766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 383/759] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:39.357988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacf5d1-176f-49a7-a542-43e232aff4dc None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:39.359235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0d856-da67-417c-880b-3015c6e843e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:39.361331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0d856-da67-417c-880b-3015c6e843e6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c9f0d856-da67-417c-880b-3015c6e843e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:39.388524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:39.389558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:39.390481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:39.390481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:39.390481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:39.401627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 377/760] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:39.406527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0d856-da67-417c-880b-3015c6e843e6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:39.482338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:39.546595 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacf5d1-176f-49a7-a542-43e232aff4dc None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:39.547451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:39.551260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0d856-da67-417c-880b-3015c6e843e6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:39.587151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dfa58-9d5d-4cd6-a7a3-19c75005fab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:39.587812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dfa58-9d5d-4cd6-a7a3-19c75005fab3 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:39.588029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dfa58-9d5d-4cd6-a7a3-19c75005fab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:39.588295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dfa58-9d5d-4cd6-a7a3-19c75005fab3 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:39.588978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dfa58-9d5d-4cd6-a7a3-19c75005fab3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:39.592066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dfa58-9d5d-4cd6-a7a3-19c75005fab3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:39.592066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dfa58-9d5d-4cd6-a7a3-19c75005fab3 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:39.610610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 384/761] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:39.617631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dfa58-9d5d-4cd6-a7a3-19c75005fab3 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:39.623111 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3edbd-b9a3-4d14-ac76-3c612f0aae08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:39.623723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3edbd-b9a3-4d14-ac76-3c612f0aae08 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:39.624001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3edbd-b9a3-4d14-ac76-3c612f0aae08 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:39.624271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3edbd-b9a3-4d14-ac76-3c612f0aae08 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:39.624915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3edbd-b9a3-4d14-ac76-3c612f0aae08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:39.625213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3edbd-b9a3-4d14-ac76-3c612f0aae08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:39.626597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3edbd-b9a3-4d14-ac76-3c612f0aae08 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:39.651337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 378/762] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:39.659668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3edbd-b9a3-4d14-ac76-3c612f0aae08 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:39.774400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3edbd-b9a3-4d14-ac76-3c612f0aae08 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:39.815109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f289b5e-16f3-40cc-9676-ef0c24e5ee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:39.815748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f289b5e-16f3-40cc-9676-ef0c24e5ee5e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:39.816030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f289b5e-16f3-40cc-9676-ef0c24e5ee5e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:39.816304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f289b5e-16f3-40cc-9676-ef0c24e5ee5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:39.829476 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f289b5e-16f3-40cc-9676-ef0c24e5ee5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:39.830952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f289b5e-16f3-40cc-9676-ef0c24e5ee5e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:39.835003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dfa58-9d5d-4cd6-a7a3-19c75005fab3 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4f289b5e-16f3-40cc-9676-ef0c24e5ee5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:39.855708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:39.856833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:39.857982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:39.857982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:39.857982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:39.857982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:39.857982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:39.857982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:39.857982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:39.857982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:39.857982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 379/763] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:39.863823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f289b5e-16f3-40cc-9676-ef0c24e5ee5e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:39.893177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:39.906344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72914-84e0-4be7-b267-1762ec1f77c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:39.906970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72914-84e0-4be7-b267-1762ec1f77c0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:39.907249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72914-84e0-4be7-b267-1762ec1f77c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:39.907519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72914-84e0-4be7-b267-1762ec1f77c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:39.908262 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca72914-84e0-4be7-b267-1762ec1f77c0 None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=b3803e9daa1848ac9feec1bdef4130de) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:39.910941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca72914-84e0-4be7-b267-1762ec1f77c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:39.910941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca72914-84e0-4be7-b267-1762ec1f77c0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:39.914112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:39.923904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 385/764] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:39 2022] GET /identity/v3/role_assignments?user.id=b3803e9daa1848ac9feec1bdef4130de&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:40.000905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f289b5e-16f3-40cc-9676-ef0c24e5ee5e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:40.038425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dc8e6-7754-44ef-b90c-eeea50e31332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:40.039041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dc8e6-7754-44ef-b90c-eeea50e31332 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:40.039321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dc8e6-7754-44ef-b90c-eeea50e31332 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:40.039596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dc8e6-7754-44ef-b90c-eeea50e31332 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:40.040963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dc8e6-7754-44ef-b90c-eeea50e31332 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:40.041494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dc8e6-7754-44ef-b90c-eeea50e31332 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:40.043046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dc8e6-7754-44ef-b90c-eeea50e31332 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:40.056110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 380/765] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:40.062826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72914-84e0-4be7-b267-1762ec1f77c0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:40.219874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72914-84e0-4be7-b267-1762ec1f77c0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:40.273772 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ca33f-8c97-42b5-bf40-6d574a4e1641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:40.274001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993df64f-e536-45d6-8207-d8a0c9b469e5 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:40.274363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993df64f-e536-45d6-8207-d8a0c9b469e5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:40.274468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca33f-8c97-42b5-bf40-6d574a4e1641 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:40.275307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca33f-8c97-42b5-bf40-6d574a4e1641 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:40.275437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993df64f-e536-45d6-8207-d8a0c9b469e5 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:40.275702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca33f-8c97-42b5-bf40-6d574a4e1641 None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/9901608b62344115aa561d00bfe88c1c/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:40.277756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ca33f-8c97-42b5-bf40-6d574a4e1641 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=9901608b62344115aa561d00bfe88c1c, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:40.279236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ca33f-8c97-42b5-bf40-6d574a4e1641 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:40.281008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 381/766] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:40.282553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccf4eb-d3d9-4bdf-b77b-247b035bdfd6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:40.282910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccf4eb-d3d9-4bdf-b77b-247b035bdfd6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:40.283257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccf4eb-d3d9-4bdf-b77b-247b035bdfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:40.302081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-98ccf4eb-d3d9-4bdf-b77b-247b035bdfd6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:40.303367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-98ccf4eb-d3d9-4bdf-b77b-247b035bdfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:40.356899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 386/767] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:30:40 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/9901608b62344115aa561d00bfe88c1c/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:30:40.386922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 382/768] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:30:40.737108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ca33f-8c97-42b5-bf40-6d574a4e1641 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:40.828084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:40.838281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958ab8-c705-4029-a929-b6e11933159a None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:40.838571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958ab8-c705-4029-a929-b6e11933159a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:40.838855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958ab8-c705-4029-a929-b6e11933159a None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:40.840530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 383/769] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:30:40.846537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6039b-3507-44e2-a3d2-95511aaa60be None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:40.846825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6039b-3507-44e2-a3d2-95511aaa60be None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:40.853382 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6039b-3507-44e2-a3d2-95511aaa60be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:40.868520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-23a6039b-3507-44e2-a3d2-95511aaa60be None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:40.870383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-23a6039b-3507-44e2-a3d2-95511aaa60be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:40.874119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:40.923870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ca33f-8c97-42b5-bf40-6d574a4e1641 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:40.952766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 384/770] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:30:41.001208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9006c1-a43f-42a5-bc5e-c69318f76235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:41.001794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9006c1-a43f-42a5-bc5e-c69318f76235 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:41.002072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9006c1-a43f-42a5-bc5e-c69318f76235 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:41.002342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9006c1-a43f-42a5-bc5e-c69318f76235 None admin] PATH_INFO: `/v3/services/network` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:41.002927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9006c1-a43f-42a5-bc5e-c69318f76235 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:41.004282 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9006c1-a43f-42a5-bc5e-c69318f76235 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-0d9006c1-a43f-42a5-bc5e-c69318f76235 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:41.018977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:41.020016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:41.020854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 387/771] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:40 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:41.026583 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6039b-3507-44e2-a3d2-95511aaa60be None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:41.197614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6039b-3507-44e2-a3d2-95511aaa60be None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:41.243833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da4579-c3ec-4377-b4bb-7b3dff41c826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:41.244994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da4579-c3ec-4377-b4bb-7b3dff41c826 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:41.245494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da4579-c3ec-4377-b4bb-7b3dff41c826 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:41.245986 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da4579-c3ec-4377-b4bb-7b3dff41c826 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:41.246863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da4579-c3ec-4377-b4bb-7b3dff41c826 None admin] RBAC: Adding query filter params (name=network) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:41.247358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da4579-c3ec-4377-b4bb-7b3dff41c826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:41.248909 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da4579-c3ec-4377-b4bb-7b3dff41c826 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:41.257906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 385/772] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:30:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:41.263131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9006c1-a43f-42a5-bc5e-c69318f76235 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:41.308951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da4579-c3ec-4377-b4bb-7b3dff41c826 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:41.315176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:41.439814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9006c1-a43f-42a5-bc5e-c69318f76235 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:41.473367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:41.499717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da4579-c3ec-4377-b4bb-7b3dff41c826 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:41.510500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7c9f-86d4-4bac-9429-2398838be9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:41.511032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7c9f-86d4-4bac-9429-2398838be9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:41.511310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7c9f-86d4-4bac-9429-2398838be9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:41.511579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7c9f-86d4-4bac-9429-2398838be9e6 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:41.512219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7c9f-86d4-4bac-9429-2398838be9e6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:41.512519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7c9f-86d4-4bac-9429-2398838be9e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:41.513922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7c9f-86d4-4bac-9429-2398838be9e6 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:41.520059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 388/773] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:30:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:41.521265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaae397-2893-4714-841b-b6515f0d3e3c None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:41.521546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaae397-2893-4714-841b-b6515f0d3e3c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:41.521919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaae397-2893-4714-841b-b6515f0d3e3c None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:41.527809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 389/774] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:41.528673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaae397-2893-4714-841b-b6515f0d3e3c None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:41.562631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db13e94-75c9-4bb3-a73d-6c9bc8727103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:41.564051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13e94-75c9-4bb3-a73d-6c9bc8727103 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:41.564816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13e94-75c9-4bb3-a73d-6c9bc8727103 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:41.565290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13e94-75c9-4bb3-a73d-6c9bc8727103 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:41.566246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db13e94-75c9-4bb3-a73d-6c9bc8727103 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:41.568356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db13e94-75c9-4bb3-a73d-6c9bc8727103 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3db13e94-75c9-4bb3-a73d-6c9bc8727103 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:41.580896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:41.581941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 03:30:41.582813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:41.585940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 386/775] 10.222.0.42 () {56 vars in 1129 bytes} [Sat Sep 17 03:30:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:41.588355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80314e13-3ec7-42aa-816c-5f517e40fc14 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:41.588872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80314e13-3ec7-42aa-816c-5f517e40fc14 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:41.589658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80314e13-3ec7-42aa-816c-5f517e40fc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:41.613091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-80314e13-3ec7-42aa-816c-5f517e40fc14 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:41.614759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-80314e13-3ec7-42aa-816c-5f517e40fc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:41.656323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaae397-2893-4714-841b-b6515f0d3e3c None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:41.694477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfca80-ab21-4e72-9019-c289f1f25a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:41.695095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfca80-ab21-4e72-9019-c289f1f25a71 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:41.695372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfca80-ab21-4e72-9019-c289f1f25a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:41.695641 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfca80-ab21-4e72-9019-c289f1f25a71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:41.696380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cfca80-ab21-4e72-9019-c289f1f25a71 None admin] RBAC: Adding query filter params (interface=public, service_id=75096fc4fe4f4324a68bc9664f50be05, region_id=RegionOne) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:41.697582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:41.697695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cfca80-ab21-4e72-9019-c289f1f25a71 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:41.700224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cfca80-ab21-4e72-9019-c289f1f25a71 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:41.704227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 387/776] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:41.705271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80314e13-3ec7-42aa-816c-5f517e40fc14 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:41.715788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 390/777] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Sep 17 03:30:41 2022] GET /identity/v3/endpoints?service_id=75096fc4fe4f4324a68bc9664f50be05&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:41.721975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfca80-ab21-4e72-9019-c289f1f25a71 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:41.820387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80314e13-3ec7-42aa-816c-5f517e40fc14 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:41.860592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e46b5a-24d4-4c73-87ca-d9fcb5616100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:41.861478 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e46b5a-24d4-4c73-87ca-d9fcb5616100 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:41.862368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e46b5a-24d4-4c73-87ca-d9fcb5616100 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:41.862806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e46b5a-24d4-4c73-87ca-d9fcb5616100 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:41.863585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e46b5a-24d4-4c73-87ca-d9fcb5616100 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:41.864037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e46b5a-24d4-4c73-87ca-d9fcb5616100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:41.865952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e46b5a-24d4-4c73-87ca-d9fcb5616100 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:41.870811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 388/778] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Sep 17 03:30:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:41.876284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e46b5a-24d4-4c73-87ca-d9fcb5616100 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:41.883043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:42.008998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:42.022516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfca80-ab21-4e72-9019-c289f1f25a71 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:42.036626 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e46b5a-24d4-4c73-87ca-d9fcb5616100 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:42.072344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea58b70-001a-4666-a21e-1ce9e91df75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:42.073987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea58b70-001a-4666-a21e-1ce9e91df75c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:42.074419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea58b70-001a-4666-a21e-1ce9e91df75c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:42.074835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea58b70-001a-4666-a21e-1ce9e91df75c None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:42.076106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea58b70-001a-4666-a21e-1ce9e91df75c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:42.078086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea58b70-001a-4666-a21e-1ce9e91df75c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:42.083677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026302f-2fde-40ce-931f-fa0d6660e41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:42.084284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026302f-2fde-40ce-931f-fa0d6660e41d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:42.084628 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026302f-2fde-40ce-931f-fa0d6660e41d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:42.085004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026302f-2fde-40ce-931f-fa0d6660e41d None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:42.085376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 391/779] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Sep 17 03:30:41 2022] GET /identity/v3/services => generated 772 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:42.085607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026302f-2fde-40ce-931f-fa0d6660e41d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:42.091378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026302f-2fde-40ce-931f-fa0d6660e41d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:42.091378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026302f-2fde-40ce-931f-fa0d6660e41d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:42.091655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea58b70-001a-4666-a21e-1ce9e91df75c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:42.098414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 389/780] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Sep 17 03:30:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:42.105089 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026302f-2fde-40ce-931f-fa0d6660e41d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:42.308217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea58b70-001a-4666-a21e-1ce9e91df75c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:42.327186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026302f-2fde-40ce-931f-fa0d6660e41d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:42.327615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:42.364029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca710b-fbe4-41ba-a1e5-87504612c72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:42.365350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca710b-fbe4-41ba-a1e5-87504612c72a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:42.365880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca710b-fbe4-41ba-a1e5-87504612c72a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:42.366351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca710b-fbe4-41ba-a1e5-87504612c72a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:42.379164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465ad35-96a6-4262-8580-1bbf98fa4c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:42.379790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465ad35-96a6-4262-8580-1bbf98fa4c1d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:42.380074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465ad35-96a6-4262-8580-1bbf98fa4c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:42.380352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465ad35-96a6-4262-8580-1bbf98fa4c1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:42.381127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465ad35-96a6-4262-8580-1bbf98fa4c1d None admin] RBAC: Adding query filter params (interface=public, service_id=9bb64bb703dc403080a680af1e7dafcb, region_id=RegionOne) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:42.381436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465ad35-96a6-4262-8580-1bbf98fa4c1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:42.383243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca710b-fbe4-41ba-a1e5-87504612c72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:42.383354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465ad35-96a6-4262-8580-1bbf98fa4c1d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-56ca710b-fbe4-41ba-a1e5-87504612c72a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:42.390051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:42.391027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:42.391876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:42.394371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 392/781] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:42.398745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 390/782] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Sep 17 03:30:42 2022] GET /identity/v3/endpoints?service_id=9bb64bb703dc403080a680af1e7dafcb&interface=public®ion_id=RegionOne => generated 194 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:42.399850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:42.400850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca710b-fbe4-41ba-a1e5-87504612c72a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:42.404393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465ad35-96a6-4262-8580-1bbf98fa4c1d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:42.519630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465ad35-96a6-4262-8580-1bbf98fa4c1d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:42.530392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca710b-fbe4-41ba-a1e5-87504612c72a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:42.555694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd943fa7-fbc4-4962-bddc-3e289fcbb3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:42.556985 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd943fa7-fbc4-4962-bddc-3e289fcbb3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:42.557295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd943fa7-fbc4-4962-bddc-3e289fcbb3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:42.557527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd943fa7-fbc4-4962-bddc-3e289fcbb3f6 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:42.558157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd943fa7-fbc4-4962-bddc-3e289fcbb3f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:42.559639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd943fa7-fbc4-4962-bddc-3e289fcbb3f6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:42.565607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 391/783] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Sep 17 03:30:42 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:42.567031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c199fa5-ffd3-4cfb-877c-dcc997cceab4 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:42.567395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c199fa5-ffd3-4cfb-877c-dcc997cceab4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:42.567672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c199fa5-ffd3-4cfb-877c-dcc997cceab4 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:42.568766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3062294-2d65-445a-b9bb-d6a86e6dc7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:42.573881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 392/784] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:42.574847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3062294-2d65-445a-b9bb-d6a86e6dc7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:42.575136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3062294-2d65-445a-b9bb-d6a86e6dc7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:42.575405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3062294-2d65-445a-b9bb-d6a86e6dc7d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:42.576124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3062294-2d65-445a-b9bb-d6a86e6dc7d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:42.576424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3062294-2d65-445a-b9bb-d6a86e6dc7d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:42.577302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2375de-3a76-439e-8ed3-41f4278c693a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:42.577839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2375de-3a76-439e-8ed3-41f4278c693a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:42.577962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3062294-2d65-445a-b9bb-d6a86e6dc7d8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:42.578419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2375de-3a76-439e-8ed3-41f4278c693a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:42.589943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 393/785] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:42.594550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3062294-2d65-445a-b9bb-d6a86e6dc7d8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:42.602416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-fc2375de-3a76-439e-8ed3-41f4278c693a None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:42.604432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fc2375de-3a76-439e-8ed3-41f4278c693a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:42.637486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:42.687574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:42.734351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 393/786] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:30:42.771276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3062294-2d65-445a-b9bb-d6a86e6dc7d8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:42.833633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a38a-21a1-4379-babf-ef812a940f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:42.834330 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a38a-21a1-4379-babf-ef812a940f4c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:42.834606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a38a-21a1-4379-babf-ef812a940f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:42.834881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a38a-21a1-4379-babf-ef812a940f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:42.846408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a38a-21a1-4379-babf-ef812a940f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:42.847862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a38a-21a1-4379-babf-ef812a940f4c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-fef5a38a-21a1-4379-babf-ef812a940f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:42.862708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:42.863677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:42.864582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:42.864582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:42.864582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:42.864582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 394/787] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:42.869683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2375de-3a76-439e-8ed3-41f4278c693a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:42.971739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2375de-3a76-439e-8ed3-41f4278c693a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:43.019763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403852f-2d7c-438f-b0f6-d934690be3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:43.020764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403852f-2d7c-438f-b0f6-d934690be3cc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:43.021375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403852f-2d7c-438f-b0f6-d934690be3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:43.021932 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403852f-2d7c-438f-b0f6-d934690be3cc None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:43.022820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403852f-2d7c-438f-b0f6-d934690be3cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:43.023365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403852f-2d7c-438f-b0f6-d934690be3cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:43.025088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403852f-2d7c-438f-b0f6-d934690be3cc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:43.043290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 394/788] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:43.049497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a38a-21a1-4379-babf-ef812a940f4c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:43.119350 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403852f-2d7c-438f-b0f6-d934690be3cc None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:43.151415 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:43.167763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a38a-21a1-4379-babf-ef812a940f4c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:43.170714 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:43.203454 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f654306-b8da-4f51-aa17-73769c24eb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:43.204189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f654306-b8da-4f51-aa17-73769c24eb0a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:43.204530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f654306-b8da-4f51-aa17-73769c24eb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:43.204861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f654306-b8da-4f51-aa17-73769c24eb0a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:43.213906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f654306-b8da-4f51-aa17-73769c24eb0a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:43.215432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f654306-b8da-4f51-aa17-73769c24eb0a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-2f654306-b8da-4f51-aa17-73769c24eb0a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:43.227032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:43.228381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:43.229581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:43.229581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:43.229581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:43.229581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:43.229581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:43.229581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:43.229581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:43.229581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:43.229581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:43.229581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:43.230353 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 395/789] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:43.231670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a5ffd-bede-448d-95f9-16c67ee60d2e None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:43.231954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a5ffd-bede-448d-95f9-16c67ee60d2e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:43.232225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a5ffd-bede-448d-95f9-16c67ee60d2e None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:43.233760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 396/790] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:43.236420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a5ffd-bede-448d-95f9-16c67ee60d2e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:43.239344 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403852f-2d7c-438f-b0f6-d934690be3cc None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:43.278268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10ce5a-1b6a-4afb-bf27-266cc0664aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:43.279033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10ce5a-1b6a-4afb-bf27-266cc0664aea None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:43.279378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10ce5a-1b6a-4afb-bf27-266cc0664aea None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:43.279753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10ce5a-1b6a-4afb-bf27-266cc0664aea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:43.293192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10ce5a-1b6a-4afb-bf27-266cc0664aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-2a10ce5a-1b6a-4afb-bf27-266cc0664aea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:43.298681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:43.299656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:43.300537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 395/791] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:43.301866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc605f-5054-4bd3-b8bb-b3a970936307 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:43.302326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc605f-5054-4bd3-b8bb-b3a970936307 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:43.302691 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc605f-5054-4bd3-b8bb-b3a970936307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:43.322493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1cfc605f-5054-4bd3-b8bb-b3a970936307 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:43.324340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1cfc605f-5054-4bd3-b8bb-b3a970936307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:43.350882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a5ffd-bede-448d-95f9-16c67ee60d2e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:43.387392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fdc7a-7a3c-4aff-83a8-90aaddad67c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:43.388177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fdc7a-7a3c-4aff-83a8-90aaddad67c3 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:43.388541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fdc7a-7a3c-4aff-83a8-90aaddad67c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:43.388918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fdc7a-7a3c-4aff-83a8-90aaddad67c3 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:43.389754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3fdc7a-7a3c-4aff-83a8-90aaddad67c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:43.390174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3fdc7a-7a3c-4aff-83a8-90aaddad67c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:43.391906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3fdc7a-7a3c-4aff-83a8-90aaddad67c3 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:43.400934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 396/792] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:43.402141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc605f-5054-4bd3-b8bb-b3a970936307 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:43.427801 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 397/793] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Sep 17 03:30:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:43.439325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fdc7a-7a3c-4aff-83a8-90aaddad67c3 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:43.439904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:43.443170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:43.547924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc605f-5054-4bd3-b8bb-b3a970936307 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:43.612111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fdc7a-7a3c-4aff-83a8-90aaddad67c3 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:43.618903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308989e-dd0f-4b05-9d38-08cad1810763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:43.619512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308989e-dd0f-4b05-9d38-08cad1810763 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:43.619788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308989e-dd0f-4b05-9d38-08cad1810763 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:43.620058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308989e-dd0f-4b05-9d38-08cad1810763 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:43.620728 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308989e-dd0f-4b05-9d38-08cad1810763 None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:43.626561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308989e-dd0f-4b05-9d38-08cad1810763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:43.627995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308989e-dd0f-4b05-9d38-08cad1810763 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:43.646866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 397/794] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:43.652907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308989e-dd0f-4b05-9d38-08cad1810763 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:43.663784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e112c-6b16-47da-ae65-c733678c5ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:43.664414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e112c-6b16-47da-ae65-c733678c5ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:43.664689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e112c-6b16-47da-ae65-c733678c5ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:43.664955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e112c-6b16-47da-ae65-c733678c5ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:43.676887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e112c-6b16-47da-ae65-c733678c5ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:43.678415 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e112c-6b16-47da-ae65-c733678c5ca7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-f56e112c-6b16-47da-ae65-c733678c5ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:43.694881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:43.695915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:43.697359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:43.697359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:43.697359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:43.697359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 398/795] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:43.702766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e112c-6b16-47da-ae65-c733678c5ca7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:43.761066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:43.808436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e112c-6b16-47da-ae65-c733678c5ca7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:43.830485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:43.840753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308989e-dd0f-4b05-9d38-08cad1810763 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:43.847122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98025279-7759-41dd-9083-e92f1adf80ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:43.847726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98025279-7759-41dd-9083-e92f1adf80ec None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:43.848036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98025279-7759-41dd-9083-e92f1adf80ec None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:43.848304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98025279-7759-41dd-9083-e92f1adf80ec None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:43.848936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98025279-7759-41dd-9083-e92f1adf80ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:43.849233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98025279-7759-41dd-9083-e92f1adf80ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:43.850632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98025279-7759-41dd-9083-e92f1adf80ec None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:43.864753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 399/796] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:43.865585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98025279-7759-41dd-9083-e92f1adf80ec None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:43.891702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa5000-9054-451c-ba1c-b0ba7a63b996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:43.892998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa5000-9054-451c-ba1c-b0ba7a63b996 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:43.893506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa5000-9054-451c-ba1c-b0ba7a63b996 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:43.893966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa5000-9054-451c-ba1c-b0ba7a63b996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:43.909978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa5000-9054-451c-ba1c-b0ba7a63b996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:43.911462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa5000-9054-451c-ba1c-b0ba7a63b996 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-51fa5000-9054-451c-ba1c-b0ba7a63b996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:43.935841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:43.936832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:43.937807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:43.937807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:43.937807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:43.943807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 398/797] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:43.943807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa5000-9054-451c-ba1c-b0ba7a63b996 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:44.001309 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98025279-7759-41dd-9083-e92f1adf80ec None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:44.021118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:44.046854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5dd8b-9017-4434-9a47-da1094beb738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:44.047871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5dd8b-9017-4434-9a47-da1094beb738 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:44.048374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5dd8b-9017-4434-9a47-da1094beb738 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:44.048846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5dd8b-9017-4434-9a47-da1094beb738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:44.062142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5dd8b-9017-4434-9a47-da1094beb738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d4e5dd8b-9017-4434-9a47-da1094beb738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:44.068993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:44.069981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:44.070836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:44.080186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 400/798] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:44.080186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5dd8b-9017-4434-9a47-da1094beb738 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:44.093230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa5000-9054-451c-ba1c-b0ba7a63b996 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:44.142101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bda9b4-1a5a-4e16-bdb2-68a45ce98da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:44.143615 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bda9b4-1a5a-4e16-bdb2-68a45ce98da3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:44.144126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bda9b4-1a5a-4e16-bdb2-68a45ce98da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:44.144614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bda9b4-1a5a-4e16-bdb2-68a45ce98da3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:44.153562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bda9b4-1a5a-4e16-bdb2-68a45ce98da3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:44.155315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bda9b4-1a5a-4e16-bdb2-68a45ce98da3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a5bda9b4-1a5a-4e16-bdb2-68a45ce98da3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:44.170899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:44.171902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:44.172815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:44.172815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:44.172815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:44.172815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:44.172815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:44.172815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:44.172815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:44.172815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:44.175754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 399/799] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:44.176773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bda9b4-1a5a-4e16-bdb2-68a45ce98da3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:44.194980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:44.282352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5dd8b-9017-4434-9a47-da1094beb738 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:44.363633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bda9b4-1a5a-4e16-bdb2-68a45ce98da3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:44.374092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100835f-5de4-4b1b-8b0c-629bd7f076db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:44.378477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100835f-5de4-4b1b-8b0c-629bd7f076db None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:44.379478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100835f-5de4-4b1b-8b0c-629bd7f076db None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:44.380944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100835f-5de4-4b1b-8b0c-629bd7f076db None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:44.381956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100835f-5de4-4b1b-8b0c-629bd7f076db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:44.384797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100835f-5de4-4b1b-8b0c-629bd7f076db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:44.390543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100835f-5de4-4b1b-8b0c-629bd7f076db None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:44.420434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e4e19-6298-4eae-a347-9156c7fb5418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:44.421096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e4e19-6298-4eae-a347-9156c7fb5418 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:44.421323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e4e19-6298-4eae-a347-9156c7fb5418 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:44.421597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e4e19-6298-4eae-a347-9156c7fb5418 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:44.425641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e4e19-6298-4eae-a347-9156c7fb5418 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:44.425641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e4e19-6298-4eae-a347-9156c7fb5418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:44.429624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e4e19-6298-4eae-a347-9156c7fb5418 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:44.433191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 401/800] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:44.434451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100835f-5de4-4b1b-8b0c-629bd7f076db None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:44.447414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 400/801] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:44.461813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e4e19-6298-4eae-a347-9156c7fb5418 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:44.496081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:44.587469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:44.625024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e4e19-6298-4eae-a347-9156c7fb5418 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:44.690571 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100835f-5de4-4b1b-8b0c-629bd7f076db None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:44.695267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5a3ca-60be-4047-b698-63248f1ea428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:44.695904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5a3ca-60be-4047-b698-63248f1ea428 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:44.696184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5a3ca-60be-4047-b698-63248f1ea428 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:44.696496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5a3ca-60be-4047-b698-63248f1ea428 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:44.707726 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5a3ca-60be-4047-b698-63248f1ea428 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:44.719535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5a3ca-60be-4047-b698-63248f1ea428 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:44.740862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869ce46-1a2f-47a3-9636-b0e78e34bdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:44.741591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ce46-1a2f-47a3-9636-b0e78e34bdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:44.741995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ce46-1a2f-47a3-9636-b0e78e34bdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:44.742343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ce46-1a2f-47a3-9636-b0e78e34bdc5 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:44.743116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869ce46-1a2f-47a3-9636-b0e78e34bdc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:44.744101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869ce46-1a2f-47a3-9636-b0e78e34bdc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:44.745654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869ce46-1a2f-47a3-9636-b0e78e34bdc5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-72d5a3ca-60be-4047-b698-63248f1ea428 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:44.748251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:44.749358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 401/802] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Sep 17 03:30:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:44.750798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5a3ca-60be-4047-b698-63248f1ea428 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:44.771121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 402/803] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:44.771121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c450e99-0763-4208-878c-4d1670a792f0 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:44.771121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c450e99-0763-4208-878c-4d1670a792f0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:44.771121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c450e99-0763-4208-878c-4d1670a792f0 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:44.771121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 403/804] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:44.771121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c450e99-0763-4208-878c-4d1670a792f0 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:44.897124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5a3ca-60be-4047-b698-63248f1ea428 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:44.903148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:44.914115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c450e99-0763-4208-878c-4d1670a792f0 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:44.935022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:44.941371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41560a3-e418-4fff-aa32-b4c701d9f7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:44.942025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41560a3-e418-4fff-aa32-b4c701d9f7bb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:44.942256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41560a3-e418-4fff-aa32-b4c701d9f7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:44.942534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41560a3-e418-4fff-aa32-b4c701d9f7bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:44.956119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41560a3-e418-4fff-aa32-b4c701d9f7bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:44.957897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41560a3-e418-4fff-aa32-b4c701d9f7bb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:44.959401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb3f36-f182-4a52-a5a7-97affe541bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:44.960291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb3f36-f182-4a52-a5a7-97affe541bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:44.961249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb3f36-f182-4a52-a5a7-97affe541bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:44.961547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb3f36-f182-4a52-a5a7-97affe541bc0 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:44.963338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb3f36-f182-4a52-a5a7-97affe541bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:44.963752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb3f36-f182-4a52-a5a7-97affe541bc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:44.965551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb3f36-f182-4a52-a5a7-97affe541bc0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e41560a3-e418-4fff-aa32-b4c701d9f7bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:44.975371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:44.976375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:44.977315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:44.977315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:44.977315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:44.981904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 402/805] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:44.982887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41560a3-e418-4fff-aa32-b4c701d9f7bb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:44.994299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 404/806] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Sep 17 03:30:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:44.996261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14a579-d3a6-49e5-b828-1bfa6c72ce81 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:44.996885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14a579-d3a6-49e5-b828-1bfa6c72ce81 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:44.997342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14a579-d3a6-49e5-b828-1bfa6c72ce81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.016162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-9e14a579-d3a6-49e5-b828-1bfa6c72ce81 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:45.017371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9e14a579-d3a6-49e5-b828-1bfa6c72ce81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:45.090715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 405/807] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:45.098634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14a579-d3a6-49e5-b828-1bfa6c72ce81 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:45.099301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41560a3-e418-4fff-aa32-b4c701d9f7bb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:45.144358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff41f4a-31a2-437f-a66c-b753bd279658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:45.145053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff41f4a-31a2-437f-a66c-b753bd279658 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:45.145338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff41f4a-31a2-437f-a66c-b753bd279658 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:45.145637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff41f4a-31a2-437f-a66c-b753bd279658 None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/c1136770f79a47f6a2af0b0a32259209/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.147569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff41f4a-31a2-437f-a66c-b753bd279658 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=c1136770f79a47f6a2af0b0a32259209, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:45.148992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff41f4a-31a2-437f-a66c-b753bd279658 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:45.207853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:45.235377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14a579-d3a6-49e5-b828-1bfa6c72ce81 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:45.236123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 403/808] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:30:44 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/c1136770f79a47f6a2af0b0a32259209/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:30:45.237218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff41f4a-31a2-437f-a66c-b753bd279658 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:45.286953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab4c56-4d04-4ca8-b4c4-30f0717f3cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:45.287947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab4c56-4d04-4ca8-b4c4-30f0717f3cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:45.288405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab4c56-4d04-4ca8-b4c4-30f0717f3cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:45.289601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab4c56-4d04-4ca8-b4c4-30f0717f3cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.290632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab4c56-4d04-4ca8-b4c4-30f0717f3cc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:45.291231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab4c56-4d04-4ca8-b4c4-30f0717f3cc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:45.293870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:45.294141 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab4c56-4d04-4ca8-b4c4-30f0717f3cc7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:45.315021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 406/809] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:45.316556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c2e71-cc37-445f-b79e-5ab3f020a7f5 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:45.316847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c2e71-cc37-445f-b79e-5ab3f020a7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:45.317154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c2e71-cc37-445f-b79e-5ab3f020a7f5 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.318626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 407/810] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:45.325470 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c2e71-cc37-445f-b79e-5ab3f020a7f5 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:45.379977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff41f4a-31a2-437f-a66c-b753bd279658 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:45.421926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf94afe-7a44-4a2c-bdf8-ea2593efc54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:45.422568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf94afe-7a44-4a2c-bdf8-ea2593efc54d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:45.422853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf94afe-7a44-4a2c-bdf8-ea2593efc54d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:45.423130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf94afe-7a44-4a2c-bdf8-ea2593efc54d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.438352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf94afe-7a44-4a2c-bdf8-ea2593efc54d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:45.439810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf94afe-7a44-4a2c-bdf8-ea2593efc54d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-caf94afe-7a44-4a2c-bdf8-ea2593efc54d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:45.455323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:45.456320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:45.457627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:45.457627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:45.457627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:45.458227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c2e71-cc37-445f-b79e-5ab3f020a7f5 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:45.461047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 404/811] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:45.462658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96bd47-ad14-4950-a56f-79aca62571aa None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:45.462953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96bd47-ad14-4950-a56f-79aca62571aa None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:45.463242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96bd47-ad14-4950-a56f-79aca62571aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.481615 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8a96bd47-ad14-4950-a56f-79aca62571aa None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:45.482839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8a96bd47-ad14-4950-a56f-79aca62571aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:45.494644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad859-a8b8-44e6-9e5a-ddd07e04bd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:45.495281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad859-a8b8-44e6-9e5a-ddd07e04bd1c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:45.495608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad859-a8b8-44e6-9e5a-ddd07e04bd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:45.495872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad859-a8b8-44e6-9e5a-ddd07e04bd1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.504500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efad859-a8b8-44e6-9e5a-ddd07e04bd1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:45.505952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efad859-a8b8-44e6-9e5a-ddd07e04bd1c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-5efad859-a8b8-44e6-9e5a-ddd07e04bd1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:45.518444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:45.519517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 408/812] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Sep 17 03:30:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:45.520547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad859-a8b8-44e6-9e5a-ddd07e04bd1c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:45.525398 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:45.570014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 405/813] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:45.570014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96bd47-ad14-4950-a56f-79aca62571aa None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:45.594424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:45.664435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad859-a8b8-44e6-9e5a-ddd07e04bd1c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:45.677475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96bd47-ad14-4950-a56f-79aca62571aa None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:45.713499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbe947-f3d4-4fe7-a3a3-686dc147c263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:45.714687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbe947-f3d4-4fe7-a3a3-686dc147c263 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:45.715135 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbe947-f3d4-4fe7-a3a3-686dc147c263 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:45.715569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbe947-f3d4-4fe7-a3a3-686dc147c263 None admin] PATH_INFO: `/v3/services/network` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.716303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbe947-f3d4-4fe7-a3a3-686dc147c263 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:45.717868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbe947-f3d4-4fe7-a3a3-686dc147c263 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:45.718288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf632eec-da45-4230-a76c-32f5715588d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:45.718893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf632eec-da45-4230-a76c-32f5715588d2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:45.719177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf632eec-da45-4230-a76c-32f5715588d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:45.719448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf632eec-da45-4230-a76c-32f5715588d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.720088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf632eec-da45-4230-a76c-32f5715588d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:45.720385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf632eec-da45-4230-a76c-32f5715588d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:45.721945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf632eec-da45-4230-a76c-32f5715588d2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-c2cbe947-f3d4-4fe7-a3a3-686dc147c263 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:45.732113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:45.733388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 03:30:45.734447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:45.737157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 409/814] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:45 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:45.737685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 406/815] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:45.738561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbe947-f3d4-4fe7-a3a3-686dc147c263 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:45.743092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf632eec-da45-4230-a76c-32f5715588d2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:45.834734 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:45.861790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:45.874048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbe947-f3d4-4fe7-a3a3-686dc147c263 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:45.880025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf632eec-da45-4230-a76c-32f5715588d2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:45.911210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd413-d30e-4364-905b-829b9bbbce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:45.911823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd413-d30e-4364-905b-829b9bbbce99 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:45.912095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd413-d30e-4364-905b-829b9bbbce99 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:45.912374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd413-d30e-4364-905b-829b9bbbce99 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.913106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bd413-d30e-4364-905b-829b9bbbce99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:45.913428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bd413-d30e-4364-905b-829b9bbbce99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:45.914876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bd413-d30e-4364-905b-829b9bbbce99 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:45.917791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a4d2f-4a25-4f11-8af5-e0fbfa1eea76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:45.918420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a4d2f-4a25-4f11-8af5-e0fbfa1eea76 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:45.918701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a4d2f-4a25-4f11-8af5-e0fbfa1eea76 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:45.918974 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a4d2f-4a25-4f11-8af5-e0fbfa1eea76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:45.925877 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a4d2f-4a25-4f11-8af5-e0fbfa1eea76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:45.927304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a4d2f-4a25-4f11-8af5-e0fbfa1eea76 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:45.933441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 410/816] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Sep 17 03:30:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:45.934446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd413-d30e-4364-905b-829b9bbbce99 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-1b1a4d2f-4a25-4f11-8af5-e0fbfa1eea76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:45.943490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:45.944407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:45.945342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:45.945342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:45.945342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:45.945342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:45.945342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:45.945342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:45.945342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:45.945342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:45.950006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 407/817] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:45.951037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a4d2f-4a25-4f11-8af5-e0fbfa1eea76 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:46.046263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd413-d30e-4364-905b-829b9bbbce99 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:46.058246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a4d2f-4a25-4f11-8af5-e0fbfa1eea76 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:46.069740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:46.083138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60624664-b9b1-422a-884d-c15d96179409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:46.084827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60624664-b9b1-422a-884d-c15d96179409 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:46.084827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60624664-b9b1-422a-884d-c15d96179409 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:46.084827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60624664-b9b1-422a-884d-c15d96179409 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:46.084827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60624664-b9b1-422a-884d-c15d96179409 None admin] RBAC: Adding query filter params (name=network) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:46.085113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60624664-b9b1-422a-884d-c15d96179409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:46.086519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60624664-b9b1-422a-884d-c15d96179409 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:46.091227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 411/818] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:30:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:46.091936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60624664-b9b1-422a-884d-c15d96179409 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:46.103494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75efce6-a75c-4674-9e4a-09350fa48954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:46.104310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75efce6-a75c-4674-9e4a-09350fa48954 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:46.104593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75efce6-a75c-4674-9e4a-09350fa48954 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:46.104871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75efce6-a75c-4674-9e4a-09350fa48954 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:46.105452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75efce6-a75c-4674-9e4a-09350fa48954 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:46.106850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75efce6-a75c-4674-9e4a-09350fa48954 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e75efce6-a75c-4674-9e4a-09350fa48954 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:46.117132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:46.118268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:46.119157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 408/819] 10.222.0.42 () {56 vars in 1129 bytes} [Sat Sep 17 03:30:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:46.120754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75efce6-a75c-4674-9e4a-09350fa48954 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:46.136011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:46.207384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60624664-b9b1-422a-884d-c15d96179409 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:46.234574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75efce6-a75c-4674-9e4a-09350fa48954 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:46.247726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359716de-d7cd-4087-8a42-8a46f6b4b765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:46.248337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359716de-d7cd-4087-8a42-8a46f6b4b765 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:46.248615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359716de-d7cd-4087-8a42-8a46f6b4b765 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:46.248888 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359716de-d7cd-4087-8a42-8a46f6b4b765 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:46.272314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124db28a-3ead-4cae-b5cc-c7b90e3b7989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:46.272906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124db28a-3ead-4cae-b5cc-c7b90e3b7989 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:46.273187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124db28a-3ead-4cae-b5cc-c7b90e3b7989 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:46.273462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124db28a-3ead-4cae-b5cc-c7b90e3b7989 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:46.274175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124db28a-3ead-4cae-b5cc-c7b90e3b7989 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:46.275029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124db28a-3ead-4cae-b5cc-c7b90e3b7989 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:46.276474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124db28a-3ead-4cae-b5cc-c7b90e3b7989 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:46.287789 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 409/820] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:46.288578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124db28a-3ead-4cae-b5cc-c7b90e3b7989 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:46.309780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:46.322818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359716de-d7cd-4087-8a42-8a46f6b4b765 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:46.325303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359716de-d7cd-4087-8a42-8a46f6b4b765 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:46.363160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:46.397543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124db28a-3ead-4cae-b5cc-c7b90e3b7989 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:46.436804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 412/821] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:30:46.436804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359716de-d7cd-4087-8a42-8a46f6b4b765 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:46.439469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89170b5a-f43c-4f28-857d-311cfc5f141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:46.440085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89170b5a-f43c-4f28-857d-311cfc5f141b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:46.440371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89170b5a-f43c-4f28-857d-311cfc5f141b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:46.440650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89170b5a-f43c-4f28-857d-311cfc5f141b None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:46.441284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89170b5a-f43c-4f28-857d-311cfc5f141b None admin] RBAC: Adding query filter params (type=network) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:46.441590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89170b5a-f43c-4f28-857d-311cfc5f141b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:46.443000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89170b5a-f43c-4f28-857d-311cfc5f141b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:46.448451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 410/822] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:30:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:46.449475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89170b5a-f43c-4f28-857d-311cfc5f141b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:46.542741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:46.559626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359716de-d7cd-4087-8a42-8a46f6b4b765 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:46.563516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89170b5a-f43c-4f28-857d-311cfc5f141b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:46.603327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24080d99-6bb1-4571-9100-28ec66884d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:46.604246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24080d99-6bb1-4571-9100-28ec66884d0a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:46.604522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24080d99-6bb1-4571-9100-28ec66884d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:46.604799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24080d99-6bb1-4571-9100-28ec66884d0a None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:46.605427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24080d99-6bb1-4571-9100-28ec66884d0a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:46.605753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24080d99-6bb1-4571-9100-28ec66884d0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:46.607127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24080d99-6bb1-4571-9100-28ec66884d0a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:46.608982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:46.617949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 413/823] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Sep 17 03:30:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:46.617949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24080d99-6bb1-4571-9100-28ec66884d0a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:46.626451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcbee7-e61c-4478-a356-16e1676698ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:46.626451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcbee7-e61c-4478-a356-16e1676698ac None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:46.626769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcbee7-e61c-4478-a356-16e1676698ac None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:46.626978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcbee7-e61c-4478-a356-16e1676698ac None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/b3803e9daa1848ac9feec1bdef4130de/roles/e33e2d7e93b4409fb2c449241a219d0d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:46.645213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bcbee7-e61c-4478-a356-16e1676698ac None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=b3803e9daa1848ac9feec1bdef4130de, role_id=e33e2d7e93b4409fb2c449241a219d0d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:46.646710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bcbee7-e61c-4478-a356-16e1676698ac None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:46.717335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24080d99-6bb1-4571-9100-28ec66884d0a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:46.724226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 411/824] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:30:46 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/b3803e9daa1848ac9feec1bdef4130de/roles/e33e2d7e93b4409fb2c449241a219d0d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:30:46.725370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcbee7-e61c-4478-a356-16e1676698ac None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:46.771178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe7ab9-20ea-45be-8b1c-602e99fc00d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:46.772291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe7ab9-20ea-45be-8b1c-602e99fc00d8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:46.772783 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe7ab9-20ea-45be-8b1c-602e99fc00d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:46.773304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe7ab9-20ea-45be-8b1c-602e99fc00d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:46.787066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe7ab9-20ea-45be-8b1c-602e99fc00d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:46.788933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe7ab9-20ea-45be-8b1c-602e99fc00d8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-e5fe7ab9-20ea-45be-8b1c-602e99fc00d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:46.806774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:46.807812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:46.809252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:46.809252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:46.809252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:46.811255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 414/825] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:46.812463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe7ab9-20ea-45be-8b1c-602e99fc00d8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:46.820727 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:46.870317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcbee7-e61c-4478-a356-16e1676698ac None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:46.924707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d372fc-b2ba-4bdd-ae2b-26db0ad04c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:46.925562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d372fc-b2ba-4bdd-ae2b-26db0ad04c1e None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:46.926034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d372fc-b2ba-4bdd-ae2b-26db0ad04c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:46.926466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d372fc-b2ba-4bdd-ae2b-26db0ad04c1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:46.927295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d372fc-b2ba-4bdd-ae2b-26db0ad04c1e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:46.928856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d372fc-b2ba-4bdd-ae2b-26db0ad04c1e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:46.954136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe7ab9-20ea-45be-8b1c-602e99fc00d8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:46.990239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01c606-a558-446c-919b-a500d2c5dd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:46.990845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c606-a558-446c-919b-a500d2c5dd7c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:46.991124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c606-a558-446c-919b-a500d2c5dd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:46.991395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c606-a558-446c-919b-a500d2c5dd7c None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:46.992020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01c606-a558-446c-919b-a500d2c5dd7c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:46.992318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01c606-a558-446c-919b-a500d2c5dd7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:46.994623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01c606-a558-446c-919b-a500d2c5dd7c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:47.000017 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 415/826] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Sep 17 03:30:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:47.001859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01c606-a558-446c-919b-a500d2c5dd7c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:47.025931 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 412/827] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:30:46 2022] POST /identity/v3/endpoints => generated 337 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:30:47.027191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d372fc-b2ba-4bdd-ae2b-26db0ad04c1e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:47.146221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:47.146693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:47.186518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01c606-a558-446c-919b-a500d2c5dd7c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:47.241032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f580215-116e-49d6-89b7-64e47a1bfb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:47.242087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f580215-116e-49d6-89b7-64e47a1bfb60 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:47.242565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f580215-116e-49d6-89b7-64e47a1bfb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:47.243031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f580215-116e-49d6-89b7-64e47a1bfb60 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:47.245105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f580215-116e-49d6-89b7-64e47a1bfb60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:47.245666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f580215-116e-49d6-89b7-64e47a1bfb60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:47.247417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f580215-116e-49d6-89b7-64e47a1bfb60 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:47.262393 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 416/828] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:47.270723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f580215-116e-49d6-89b7-64e47a1bfb60 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:47.304363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d372fc-b2ba-4bdd-ae2b-26db0ad04c1e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:47.363492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d79635-57bf-4219-abcd-462c880023ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:47.363967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d79635-57bf-4219-abcd-462c880023ce None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:47.364339 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d79635-57bf-4219-abcd-462c880023ce None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:47.364617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d79635-57bf-4219-abcd-462c880023ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:47.365293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d79635-57bf-4219-abcd-462c880023ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:47.366929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d79635-57bf-4219-abcd-462c880023ce None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:47.427634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f580215-116e-49d6-89b7-64e47a1bfb60 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:47.450774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 413/829] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:30:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:30:47.458210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:47.474681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27460fac-5a95-4ae5-8b1f-785b29d9c74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:47.475704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27460fac-5a95-4ae5-8b1f-785b29d9c74c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:47.476146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27460fac-5a95-4ae5-8b1f-785b29d9c74c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:47.476691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27460fac-5a95-4ae5-8b1f-785b29d9c74c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:47.486001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27460fac-5a95-4ae5-8b1f-785b29d9c74c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:47.487899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27460fac-5a95-4ae5-8b1f-785b29d9c74c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-27460fac-5a95-4ae5-8b1f-785b29d9c74c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:47.502342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:47.503395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:47.504368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:47.504368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:47.504368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:47.504368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:47.504368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:47.504368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:47.504368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:47.504368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:47.507048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 417/830] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:47.514294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d79635-57bf-4219-abcd-462c880023ce None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:47.535000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3dbae-d1b4-4de8-9e10-fe50a21a1fe8 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:47.535355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3dbae-d1b4-4de8-9e10-fe50a21a1fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:47.535640 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3dbae-d1b4-4de8-9e10-fe50a21a1fe8 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:47.538151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 418/831] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:30:47.543690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4c879-11b0-4540-a6b6-368dba83a744 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:47.543976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4c879-11b0-4540-a6b6-368dba83a744 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:47.544442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4c879-11b0-4540-a6b6-368dba83a744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:47.572202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ffb4c879-11b0-4540-a6b6-368dba83a744 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:47.574292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ffb4c879-11b0-4540-a6b6-368dba83a744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:47.648683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:47.659111 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d79635-57bf-4219-abcd-462c880023ce None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:47.659314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 419/832] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:47.698167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0255e-499b-40cc-a44a-b057f8fee5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:47.698954 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0255e-499b-40cc-a44a-b057f8fee5ec None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:47.699230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0255e-499b-40cc-a44a-b057f8fee5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:47.699497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0255e-499b-40cc-a44a-b057f8fee5ec None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:47.700187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0255e-499b-40cc-a44a-b057f8fee5ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:47.700487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0255e-499b-40cc-a44a-b057f8fee5ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:47.702037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0255e-499b-40cc-a44a-b057f8fee5ec None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:47.715441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 414/833] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:47.722670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4c879-11b0-4540-a6b6-368dba83a744 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:47.963284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0255e-499b-40cc-a44a-b057f8fee5ec None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:47.967434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4c879-11b0-4540-a6b6-368dba83a744 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:47.979457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:48.003422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bdad1-f113-40cc-a9cd-dfac7da857f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:48.004032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bdad1-f113-40cc-a9cd-dfac7da857f3 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:48.004314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bdad1-f113-40cc-a9cd-dfac7da857f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:48.004978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bdad1-f113-40cc-a9cd-dfac7da857f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:48.005292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bdad1-f113-40cc-a9cd-dfac7da857f3 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=9901608b62344115aa561d00bfe88c1c) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:48.005599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bdad1-f113-40cc-a9cd-dfac7da857f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:48.007077 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bdad1-f113-40cc-a9cd-dfac7da857f3 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:48.017904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 420/834] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:47 2022] GET /identity/v3/role_assignments?user.id=9901608b62344115aa561d00bfe88c1c&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:48.087119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0255e-499b-40cc-a44a-b057f8fee5ec None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:48.125428 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69633e5b-9dd9-4ba1-b95a-ab774e1b76bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:48.126595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69633e5b-9dd9-4ba1-b95a-ab774e1b76bb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:48.127076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69633e5b-9dd9-4ba1-b95a-ab774e1b76bb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:48.127545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69633e5b-9dd9-4ba1-b95a-ab774e1b76bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:48.140156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69633e5b-9dd9-4ba1-b95a-ab774e1b76bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-69633e5b-9dd9-4ba1-b95a-ab774e1b76bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:48.146682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:48.147687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:30:48.148541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:48.151514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 415/835] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:48.158875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bdad1-f113-40cc-a9cd-dfac7da857f3 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:48.278592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bdad1-f113-40cc-a9cd-dfac7da857f3 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:48.315978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c54e2-0993-4d01-9d25-adff792c6af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:48.316737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c54e2-0993-4d01-9d25-adff792c6af8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:48.317110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c54e2-0993-4d01-9d25-adff792c6af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:48.317478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c54e2-0993-4d01-9d25-adff792c6af8 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:48.318346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c54e2-0993-4d01-9d25-adff792c6af8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:48.318749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c54e2-0993-4d01-9d25-adff792c6af8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:48.320218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c54e2-0993-4d01-9d25-adff792c6af8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:48.332441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 421/836] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:30:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:48.338394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69633e5b-9dd9-4ba1-b95a-ab774e1b76bb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:48.360091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:48.458329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69633e5b-9dd9-4ba1-b95a-ab774e1b76bb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:48.499686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44aa10-dbf0-4c5d-9653-17240914838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:48.500316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44aa10-dbf0-4c5d-9653-17240914838e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:48.500591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44aa10-dbf0-4c5d-9653-17240914838e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:48.500860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44aa10-dbf0-4c5d-9653-17240914838e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:48.512102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44aa10-dbf0-4c5d-9653-17240914838e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:48.513642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44aa10-dbf0-4c5d-9653-17240914838e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-da44aa10-dbf0-4c5d-9653-17240914838e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:48.530861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:48.532155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:48.533091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:48.533091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:48.533091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:48.533091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 416/837] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:48.536673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c54e2-0993-4d01-9d25-adff792c6af8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:48.586191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:48.648617 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c54e2-0993-4d01-9d25-adff792c6af8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:48.686367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bffc8-8837-4398-9728-d4bae5daf052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:48.687361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bffc8-8837-4398-9728-d4bae5daf052 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:48.687844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bffc8-8837-4398-9728-d4bae5daf052 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:48.688330 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bffc8-8837-4398-9728-d4bae5daf052 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:48.689167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bffc8-8837-4398-9728-d4bae5daf052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:48.689711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bffc8-8837-4398-9728-d4bae5daf052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:48.691339 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bffc8-8837-4398-9728-d4bae5daf052 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:48.705034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 422/838] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:48.713868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44aa10-dbf0-4c5d-9653-17240914838e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:48.782162 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:48.820257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44aa10-dbf0-4c5d-9653-17240914838e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:48.858037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09139836-916e-4745-b21e-67f298f75553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:48.858037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09139836-916e-4745-b21e-67f298f75553 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:48.858037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09139836-916e-4745-b21e-67f298f75553 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:48.858364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09139836-916e-4745-b21e-67f298f75553 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:48.870936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09139836-916e-4745-b21e-67f298f75553 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:48.872411 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09139836-916e-4745-b21e-67f298f75553 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-09139836-916e-4745-b21e-67f298f75553 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:48.885015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:48.886129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:48.887549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 417/839] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:30:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:48.892470 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bffc8-8837-4398-9728-d4bae5daf052 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:49.002481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bffc8-8837-4398-9728-d4bae5daf052 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:49.006054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:49.041842 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4b76-7ad8-40c7-a369-3f1bd1300f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:49.042866 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4b76-7ad8-40c7-a369-3f1bd1300f23 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:49.043351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4b76-7ad8-40c7-a369-3f1bd1300f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:49.043867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4b76-7ad8-40c7-a369-3f1bd1300f23 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:49.044827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa4b76-7ad8-40c7-a369-3f1bd1300f23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:49.045350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa4b76-7ad8-40c7-a369-3f1bd1300f23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:49.047048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa4b76-7ad8-40c7-a369-3f1bd1300f23 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:49.066821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 423/840] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Sep 17 03:30:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:49.077885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09139836-916e-4745-b21e-67f298f75553 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:49.096561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3164bf-6ff6-4948-b7dd-00a8d9b4b36e None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:49.097196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3164bf-6ff6-4948-b7dd-00a8d9b4b36e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:49.097693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3164bf-6ff6-4948-b7dd-00a8d9b4b36e None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:49.099600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 424/841] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:30:49.107069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93024ad5-cfc3-42ec-9a23-2682ba5b423b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:49.107348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93024ad5-cfc3-42ec-9a23-2682ba5b423b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:49.107620 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93024ad5-cfc3-42ec-9a23-2682ba5b423b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:49.128546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-93024ad5-cfc3-42ec-9a23-2682ba5b423b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:49.129715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-93024ad5-cfc3-42ec-9a23-2682ba5b423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:49.207793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09139836-916e-4745-b21e-67f298f75553 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:49.217542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 425/842] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:30:49.241652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:49.250652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1dce6-2064-49d2-b9d3-ee78e90b1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:49.251411 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1dce6-2064-49d2-b9d3-ee78e90b1d2c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:49.251830 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1dce6-2064-49d2-b9d3-ee78e90b1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:49.259733 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1dce6-2064-49d2-b9d3-ee78e90b1d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:49.323567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1dce6-2064-49d2-b9d3-ee78e90b1d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:49.325427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1dce6-2064-49d2-b9d3-ee78e90b1d2c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:49.433771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 418/843] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:49 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:30:49.438972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93024ad5-cfc3-42ec-9a23-2682ba5b423b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:49.536605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1dce6-2064-49d2-b9d3-ee78e90b1d2c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:49.598906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:49.605606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:49.626543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93024ad5-cfc3-42ec-9a23-2682ba5b423b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:49.690482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12960069-3380-4387-a2d1-f1d8d330391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:49.691491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12960069-3380-4387-a2d1-f1d8d330391f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:49.692301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12960069-3380-4387-a2d1-f1d8d330391f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:49.692767 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12960069-3380-4387-a2d1-f1d8d330391f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:49.717750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12960069-3380-4387-a2d1-f1d8d330391f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:49.717750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12960069-3380-4387-a2d1-f1d8d330391f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-12960069-3380-4387-a2d1-f1d8d330391f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:49.740709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:49.741695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:49.742643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:49.742643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:49.742643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:49.747845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 426/844] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:49.753749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12960069-3380-4387-a2d1-f1d8d330391f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:49.760376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1dce6-2064-49d2-b9d3-ee78e90b1d2c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:49.842581 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd20de-c73e-4a7f-979b-5a14d19ca18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:49.843602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd20de-c73e-4a7f-979b-5a14d19ca18e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:49.844101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd20de-c73e-4a7f-979b-5a14d19ca18e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:49.844580 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd20de-c73e-4a7f-979b-5a14d19ca18e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:49.863285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd20de-c73e-4a7f-979b-5a14d19ca18e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-34fd20de-c73e-4a7f-979b-5a14d19ca18e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:49.879826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:49.880815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:30:49.881683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:49.890687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 419/845] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:30:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:49.894871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd20de-c73e-4a7f-979b-5a14d19ca18e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:49.962345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12960069-3380-4387-a2d1-f1d8d330391f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:50.007626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f866a-8667-4cf4-942f-58b82dce3870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:50.007626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f866a-8667-4cf4-942f-58b82dce3870 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.007626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f866a-8667-4cf4-942f-58b82dce3870 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.007626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f866a-8667-4cf4-942f-58b82dce3870 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.008961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f866a-8667-4cf4-942f-58b82dce3870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:50.008961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f866a-8667-4cf4-942f-58b82dce3870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:50.011036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f866a-8667-4cf4-942f-58b82dce3870 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:50.018129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:50.019953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:50.032357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 427/846] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:50.032357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563b2f0-d668-4307-ad88-3b1823748438 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.032357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563b2f0-d668-4307-ad88-3b1823748438 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.032357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563b2f0-d668-4307-ad88-3b1823748438 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.032357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 428/847] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:50.038179 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563b2f0-d668-4307-ad88-3b1823748438 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:50.041216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd20de-c73e-4a7f-979b-5a14d19ca18e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:50.087630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2affe-6d15-427c-b361-ca238ff4762d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:50.088367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2affe-6d15-427c-b361-ca238ff4762d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.088571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2affe-6d15-427c-b361-ca238ff4762d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.088850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2affe-6d15-427c-b361-ca238ff4762d None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.089552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2affe-6d15-427c-b361-ca238ff4762d None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:50.089833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2affe-6d15-427c-b361-ca238ff4762d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:50.092076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2affe-6d15-427c-b361-ca238ff4762d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:50.101708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 420/848] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:30:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:50.103088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a896a44-c45a-4f2e-b89c-b2ffae1a3e56 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.103380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a896a44-c45a-4f2e-b89c-b2ffae1a3e56 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.103660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a896a44-c45a-4f2e-b89c-b2ffae1a3e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.123027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6a896a44-c45a-4f2e-b89c-b2ffae1a3e56 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:50.124124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6a896a44-c45a-4f2e-b89c-b2ffae1a3e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:50.178028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563b2f0-d668-4307-ad88-3b1823748438 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:50.207340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 421/849] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:50.208634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a896a44-c45a-4f2e-b89c-b2ffae1a3e56 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:50.258345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ba0c5-994e-48ba-85f7-73095abdcd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:50.259596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ba0c5-994e-48ba-85f7-73095abdcd6c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.259921 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ba0c5-994e-48ba-85f7-73095abdcd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.260228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ba0c5-994e-48ba-85f7-73095abdcd6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.271426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ba0c5-994e-48ba-85f7-73095abdcd6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:50.272917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ba0c5-994e-48ba-85f7-73095abdcd6c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-e46ba0c5-994e-48ba-85f7-73095abdcd6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:50.294450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:50.295457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:50.296376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:50.296376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:50.296376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:50.296376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:50.296376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:50.296376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:50.296376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:50.296376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:50.308299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 429/850] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:50.309276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ba0c5-994e-48ba-85f7-73095abdcd6c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:50.343584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:50.393267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a896a44-c45a-4f2e-b89c-b2ffae1a3e56 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:50.418482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:50.439739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3445da-3eb7-4851-ae87-b52ce3d894e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:50.440331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3445da-3eb7-4851-ae87-b52ce3d894e2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.440660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3445da-3eb7-4851-ae87-b52ce3d894e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.440935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3445da-3eb7-4851-ae87-b52ce3d894e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.456436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3445da-3eb7-4851-ae87-b52ce3d894e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:50.457912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3445da-3eb7-4851-ae87-b52ce3d894e2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:50.472465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ba0c5-994e-48ba-85f7-73095abdcd6c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3c3445da-3eb7-4851-ae87-b52ce3d894e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:50.476494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:50.478174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:50.479180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:50.479180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:50.479180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:50.483261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 422/851] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:50.484930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b70c5-02b2-480e-a6fb-b346eff670dd None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.485310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b70c5-02b2-480e-a6fb-b346eff670dd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.485592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b70c5-02b2-480e-a6fb-b346eff670dd None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.487641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 423/852] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:50.490751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b70c5-02b2-480e-a6fb-b346eff670dd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:50.513485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665ed48-f023-48f5-8333-4d30736aa0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:50.514279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ed48-f023-48f5-8333-4d30736aa0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.514528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ed48-f023-48f5-8333-4d30736aa0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.514792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ed48-f023-48f5-8333-4d30736aa0f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.515485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665ed48-f023-48f5-8333-4d30736aa0f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:50.515793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665ed48-f023-48f5-8333-4d30736aa0f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:50.517250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665ed48-f023-48f5-8333-4d30736aa0f4 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:50.530337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 430/853] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:50.531758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206f4bf-59a0-47c7-95b8-fdb6c5def859 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.532038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206f4bf-59a0-47c7-95b8-fdb6c5def859 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.532300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206f4bf-59a0-47c7-95b8-fdb6c5def859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.550042 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-1206f4bf-59a0-47c7-95b8-fdb6c5def859 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:50.551337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1206f4bf-59a0-47c7-95b8-fdb6c5def859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:50.607547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b70c5-02b2-480e-a6fb-b346eff670dd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:50.624787 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 431/854] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:50.625887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206f4bf-59a0-47c7-95b8-fdb6c5def859 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:50.649286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:50.660673 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad6c01-a699-40ce-a347-968079497e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:50.661858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6c01-a699-40ce-a347-968079497e66 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.662656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6c01-a699-40ce-a347-968079497e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.663066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6c01-a699-40ce-a347-968079497e66 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.664787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad6c01-a699-40ce-a347-968079497e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:50.665988 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad6c01-a699-40ce-a347-968079497e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:50.668431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad6c01-a699-40ce-a347-968079497e66 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:50.683649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:50.688338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 424/855] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:50.688338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad6c01-a699-40ce-a347-968079497e66 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:50.746557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206f4bf-59a0-47c7-95b8-fdb6c5def859 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:50.790719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d1ccb-72fa-407f-8e2a-0c50a26272d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:50.791424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1ccb-72fa-407f-8e2a-0c50a26272d8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.791779 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1ccb-72fa-407f-8e2a-0c50a26272d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.792135 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1ccb-72fa-407f-8e2a-0c50a26272d8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.792760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d1ccb-72fa-407f-8e2a-0c50a26272d8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:50.796597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d1ccb-72fa-407f-8e2a-0c50a26272d8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-522d1ccb-72fa-407f-8e2a-0c50a26272d8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:50.808352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:50.809533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:50.810441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 432/856] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:30:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:50.811252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d1ccb-72fa-407f-8e2a-0c50a26272d8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:50.861384 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad6c01-a699-40ce-a347-968079497e66 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:50.908622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7ca43-8574-4ac1-a204-771eb41b21c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:50.909253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7ca43-8574-4ac1-a204-771eb41b21c3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.909533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7ca43-8574-4ac1-a204-771eb41b21c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.909862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7ca43-8574-4ac1-a204-771eb41b21c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:50.910566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7ca43-8574-4ac1-a204-771eb41b21c3 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=c1136770f79a47f6a2af0b0a32259209) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:50.910874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7ca43-8574-4ac1-a204-771eb41b21c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:50.912300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7ca43-8574-4ac1-a204-771eb41b21c3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:50.918594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 425/857] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:50 2022] GET /identity/v3/role_assignments?user.id=c1136770f79a47f6a2af0b0a32259209&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:50.919397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7ca43-8574-4ac1-a204-771eb41b21c3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:50.943918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d1ccb-72fa-407f-8e2a-0c50a26272d8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:50.958974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:50.989822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4a003-4fee-4b53-a6bc-ee5b65536278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:50.990860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4a003-4fee-4b53-a6bc-ee5b65536278 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:50.991134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4a003-4fee-4b53-a6bc-ee5b65536278 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:50.991531 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4a003-4fee-4b53-a6bc-ee5b65536278 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:51.002233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4a003-4fee-4b53-a6bc-ee5b65536278 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:51.003810 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4a003-4fee-4b53-a6bc-ee5b65536278 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-eaf4a003-4fee-4b53-a6bc-ee5b65536278 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:51.017000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:51.018255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:51.019356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 433/858] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Sep 17 03:30:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:51.019970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4a003-4fee-4b53-a6bc-ee5b65536278 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:51.038231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:51.100888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7ca43-8574-4ac1-a204-771eb41b21c3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:51.148209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6960b-c6ab-4615-b3bc-b79e7e9ae7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:51.148818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6960b-c6ab-4615-b3bc-b79e7e9ae7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:51.149094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6960b-c6ab-4615-b3bc-b79e7e9ae7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:51.149586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6960b-c6ab-4615-b3bc-b79e7e9ae7f5 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:51.150267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6960b-c6ab-4615-b3bc-b79e7e9ae7f5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:51.150574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6960b-c6ab-4615-b3bc-b79e7e9ae7f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:51.151998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6960b-c6ab-4615-b3bc-b79e7e9ae7f5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:51.159732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 426/859] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:30:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:51.161840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6960b-c6ab-4615-b3bc-b79e7e9ae7f5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:51.192859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4a003-4fee-4b53-a6bc-ee5b65536278 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:51.233834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc3b31-d59f-4296-a9ce-62154032cb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:51.234458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc3b31-d59f-4296-a9ce-62154032cb83 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:51.234722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc3b31-d59f-4296-a9ce-62154032cb83 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:51.235002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc3b31-d59f-4296-a9ce-62154032cb83 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:51.235565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc3b31-d59f-4296-a9ce-62154032cb83 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:51.236906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc3b31-d59f-4296-a9ce-62154032cb83 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d2bc3b31-d59f-4296-a9ce-62154032cb83 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:51.245938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:51.246930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:51.247851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 434/860] 10.222.0.42 () {56 vars in 1132 bytes} [Sat Sep 17 03:30:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:51.248761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc3b31-d59f-4296-a9ce-62154032cb83 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:51.291663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:51.359005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6960b-c6ab-4615-b3bc-b79e7e9ae7f5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:51.388297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc3b31-d59f-4296-a9ce-62154032cb83 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:51.388513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:51.408304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058e60-7381-4ce7-8353-42eea2885235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:51.409226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058e60-7381-4ce7-8353-42eea2885235 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:51.409555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058e60-7381-4ce7-8353-42eea2885235 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:51.409914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058e60-7381-4ce7-8353-42eea2885235 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:51.410589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0058e60-7381-4ce7-8353-42eea2885235 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:51.410910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0058e60-7381-4ce7-8353-42eea2885235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:51.412403 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0058e60-7381-4ce7-8353-42eea2885235 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:51.435549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 427/861] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Sep 17 03:30:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:51.436666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058e60-7381-4ce7-8353-42eea2885235 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:51.441299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde7c9d-0612-4916-bbd5-fcbfb176f04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:51.441938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde7c9d-0612-4916-bbd5-fcbfb176f04b None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:51.442212 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde7c9d-0612-4916-bbd5-fcbfb176f04b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:51.446618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde7c9d-0612-4916-bbd5-fcbfb176f04b None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:51.446618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde7c9d-0612-4916-bbd5-fcbfb176f04b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:51.446618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde7c9d-0612-4916-bbd5-fcbfb176f04b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:51.446618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde7c9d-0612-4916-bbd5-fcbfb176f04b None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:51.450317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 435/862] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:30:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:51.451122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde7c9d-0612-4916-bbd5-fcbfb176f04b None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:51.574415 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:51.577026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058e60-7381-4ce7-8353-42eea2885235 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:51.579377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde7c9d-0612-4916-bbd5-fcbfb176f04b None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:51.628290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdd9fa-e98c-49b7-ad36-b787db96e17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:51.628786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f2826-3e76-4979-a1d0-8143e3394ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:51.629395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f2826-3e76-4979-a1d0-8143e3394ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:51.629673 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f2826-3e76-4979-a1d0-8143e3394ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:51.629976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f2826-3e76-4979-a1d0-8143e3394ba9 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:51.630224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdd9fa-e98c-49b7-ad36-b787db96e17e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:51.630520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdd9fa-e98c-49b7-ad36-b787db96e17e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:51.630621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f2826-3e76-4979-a1d0-8143e3394ba9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:51.630787 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdd9fa-e98c-49b7-ad36-b787db96e17e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:51.630881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f2826-3e76-4979-a1d0-8143e3394ba9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:51.632240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f2826-3e76-4979-a1d0-8143e3394ba9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:51.638675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 428/863] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:30:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:51.644790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f2826-3e76-4979-a1d0-8143e3394ba9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:51.696867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:51.721166 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdd9fa-e98c-49b7-ad36-b787db96e17e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:51.723157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdd9fa-e98c-49b7-ad36-b787db96e17e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:51.765671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f2826-3e76-4979-a1d0-8143e3394ba9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:51.807835 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ca8e7-35c1-4e0a-830b-871bf4403701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:51.808713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ca8e7-35c1-4e0a-830b-871bf4403701 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:51.809240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ca8e7-35c1-4e0a-830b-871bf4403701 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:51.809617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ca8e7-35c1-4e0a-830b-871bf4403701 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:51.811247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ca8e7-35c1-4e0a-830b-871bf4403701 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:51.811661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ca8e7-35c1-4e0a-830b-871bf4403701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:51.813902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ca8e7-35c1-4e0a-830b-871bf4403701 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:51.825762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 429/864] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:30:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:51.865643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 436/865] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:30:51.866234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:51.874285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ca8e7-35c1-4e0a-830b-871bf4403701 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:51.999200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ca8e7-35c1-4e0a-830b-871bf4403701 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:52.002137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:52.044634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0345c2-3ecd-4080-ab5d-3c557b338cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:52.045297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0345c2-3ecd-4080-ab5d-3c557b338cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:52.045576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0345c2-3ecd-4080-ab5d-3c557b338cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:52.045936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0345c2-3ecd-4080-ab5d-3c557b338cf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:52.058960 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0345c2-3ecd-4080-ab5d-3c557b338cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:52.060415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0345c2-3ecd-4080-ab5d-3c557b338cf5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-1b0345c2-3ecd-4080-ab5d-3c557b338cf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:52.078210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:52.079213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:52.080129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:52.080129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:52.080129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:52.082328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 430/866] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:52.088457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdd9fa-e98c-49b7-ad36-b787db96e17e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:52.194078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdd9fa-e98c-49b7-ad36-b787db96e17e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:52.230660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331518e-9d24-4594-900f-71da1a3f3a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:52.231335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331518e-9d24-4594-900f-71da1a3f3a20 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:52.231612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331518e-9d24-4594-900f-71da1a3f3a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:52.231886 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331518e-9d24-4594-900f-71da1a3f3a20 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:52.232534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331518e-9d24-4594-900f-71da1a3f3a20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:52.232838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331518e-9d24-4594-900f-71da1a3f3a20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:52.234248 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331518e-9d24-4594-900f-71da1a3f3a20 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:52.247277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 437/867] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:52.252859 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0345c2-3ecd-4080-ab5d-3c557b338cf5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:52.363901 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0345c2-3ecd-4080-ab5d-3c557b338cf5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:52.402273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4994b-494c-4991-bdee-b5b405177097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:52.402918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4994b-494c-4991-bdee-b5b405177097 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:52.405767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4994b-494c-4991-bdee-b5b405177097 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:52.405767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4994b-494c-4991-bdee-b5b405177097 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:52.410907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4994b-494c-4991-bdee-b5b405177097 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:52.412329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4994b-494c-4991-bdee-b5b405177097 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-03d4994b-494c-4991-bdee-b5b405177097 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:52.423699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:52.424621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:52.425537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:52.425537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:52.425537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:52.425537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:52.425537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:52.425537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:52.425537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:52.425537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:52.425537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 431/868] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:52.433252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331518e-9d24-4594-900f-71da1a3f3a20 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:52.500570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:52.542968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331518e-9d24-4594-900f-71da1a3f3a20 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:52.578169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a92aa-5317-4c82-9f77-ceb766515fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:52.579091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a92aa-5317-4c82-9f77-ceb766515fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:52.579586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a92aa-5317-4c82-9f77-ceb766515fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:52.580050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a92aa-5317-4c82-9f77-ceb766515fe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:52.580915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a92aa-5317-4c82-9f77-ceb766515fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:52.581422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a92aa-5317-4c82-9f77-ceb766515fe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:52.583069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a92aa-5317-4c82-9f77-ceb766515fe2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:52.597419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 438/869] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:52.606822 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4994b-494c-4991-bdee-b5b405177097 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:52.631602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:52.730534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4994b-494c-4991-bdee-b5b405177097 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:52.766507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f72f8-92f2-48be-a349-abbfd942380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:52.767108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f72f8-92f2-48be-a349-abbfd942380c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:52.767386 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f72f8-92f2-48be-a349-abbfd942380c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:52.767657 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f72f8-92f2-48be-a349-abbfd942380c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:52.768376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f72f8-92f2-48be-a349-abbfd942380c None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=b3803e9daa1848ac9feec1bdef4130de) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:52.768680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f72f8-92f2-48be-a349-abbfd942380c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:52.770120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f72f8-92f2-48be-a349-abbfd942380c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:52.776412 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 432/870] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:52 2022] GET /identity/v3/role_assignments?user.id=b3803e9daa1848ac9feec1bdef4130de&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:53.235365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968badd-141b-414c-8344-5ef6b286991f None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:53.235706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968badd-141b-414c-8344-5ef6b286991f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:53.236037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968badd-141b-414c-8344-5ef6b286991f None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:53.237684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 439/871] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:53.243671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297760a-9099-486a-88df-1bd36d7e6e70 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:53.244268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297760a-9099-486a-88df-1bd36d7e6e70 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:53.244268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297760a-9099-486a-88df-1bd36d7e6e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:53.263042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c297760a-9099-486a-88df-1bd36d7e6e70 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:53.264234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c297760a-9099-486a-88df-1bd36d7e6e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:53.345884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 433/872] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:30:53.622239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968badd-141b-414c-8344-5ef6b286991f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:53.724324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da054074-2967-49df-8e72-4cd5deea6c4a None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:53.724652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da054074-2967-49df-8e72-4cd5deea6c4a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:53.724910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da054074-2967-49df-8e72-4cd5deea6c4a None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:53.726445 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 434/873] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:53.731836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9f042-61ed-4e74-bd30-e10fce6b30cd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:53.732077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9f042-61ed-4e74-bd30-e10fce6b30cd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:53.732313 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9f042-61ed-4e74-bd30-e10fce6b30cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:53.733844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968badd-141b-414c-8344-5ef6b286991f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:53.752951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:53.754847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a6f9f042-61ed-4e74-bd30-e10fce6b30cd None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:53.756990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a6f9f042-61ed-4e74-bd30-e10fce6b30cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:53.760567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:53.771169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddeacb-56ba-4512-93e7-27ae86a03bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:53.772142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddeacb-56ba-4512-93e7-27ae86a03bde None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:53.772142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddeacb-56ba-4512-93e7-27ae86a03bde None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:53.772363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddeacb-56ba-4512-93e7-27ae86a03bde None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:53.772935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddeacb-56ba-4512-93e7-27ae86a03bde None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:53.775025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddeacb-56ba-4512-93e7-27ae86a03bde None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-b6ddeacb-56ba-4512-93e7-27ae86a03bde None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:53.784670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:53.785833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:30:53.786716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:53.791231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 440/874] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:30:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:53.792654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddeacb-56ba-4512-93e7-27ae86a03bde None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:53.859990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 435/875] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:30:53.909228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddeacb-56ba-4512-93e7-27ae86a03bde None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:53.943740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe628-f70e-44a9-b5c8-c1a0ff0c01f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:53.944351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe628-f70e-44a9-b5c8-c1a0ff0c01f9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:53.944635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe628-f70e-44a9-b5c8-c1a0ff0c01f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:53.944914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe628-f70e-44a9-b5c8-c1a0ff0c01f9 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:53.945545 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbe628-f70e-44a9-b5c8-c1a0ff0c01f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:53.945885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbe628-f70e-44a9-b5c8-c1a0ff0c01f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:53.947568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbe628-f70e-44a9-b5c8-c1a0ff0c01f9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:53.957973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 441/876] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Sep 17 03:30:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:53.960434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9f042-61ed-4e74-bd30-e10fce6b30cd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:54.075899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9f042-61ed-4e74-bd30-e10fce6b30cd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:54.120007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a147596-728e-4f5a-afcf-4ce0a2ef2319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:54.120615 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a147596-728e-4f5a-afcf-4ce0a2ef2319 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:54.120894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a147596-728e-4f5a-afcf-4ce0a2ef2319 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:54.121164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a147596-728e-4f5a-afcf-4ce0a2ef2319 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:54.121837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a147596-728e-4f5a-afcf-4ce0a2ef2319 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:54.122140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a147596-728e-4f5a-afcf-4ce0a2ef2319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:54.123502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a147596-728e-4f5a-afcf-4ce0a2ef2319 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:54.128664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 436/877] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Sep 17 03:30:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:54.143189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe628-f70e-44a9-b5c8-c1a0ff0c01f9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:54.156227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5e3b7-11c0-44fe-ae78-ebc6852b85fd None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:54.156571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5e3b7-11c0-44fe-ae78-ebc6852b85fd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:54.156788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5e3b7-11c0-44fe-ae78-ebc6852b85fd None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:54.158566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 437/878] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:30:54.164256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fafbd-ea8b-465a-b9c4-d0b76ec564bf None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:54.164532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fafbd-ea8b-465a-b9c4-d0b76ec564bf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:54.164733 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fafbd-ea8b-465a-b9c4-d0b76ec564bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:54.173704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:54.189024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3d0fafbd-ea8b-465a-b9c4-d0b76ec564bf None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:54.190357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3d0fafbd-ea8b-465a-b9c4-d0b76ec564bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:54.202543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:54.278740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe628-f70e-44a9-b5c8-c1a0ff0c01f9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:54.290298 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 438/879] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:30:54.333015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcbbd3-fcb6-4f5a-b842-8d78bed2c30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:54.333676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcbbd3-fcb6-4f5a-b842-8d78bed2c30d None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:54.334051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcbbd3-fcb6-4f5a-b842-8d78bed2c30d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:54.334324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcbbd3-fcb6-4f5a-b842-8d78bed2c30d None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:54.334975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcbbd3-fcb6-4f5a-b842-8d78bed2c30d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:54.336336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcbbd3-fcb6-4f5a-b842-8d78bed2c30d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:54.543478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 442/880] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:30:54 2022] POST /identity/v3/services => generated 251 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:30:54.576888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fafbd-ea8b-465a-b9c4-d0b76ec564bf None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:54.710667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fafbd-ea8b-465a-b9c4-d0b76ec564bf None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:54.748946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:54.759913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4a351-615c-4389-8317-655ed0ebd2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:54.761112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4a351-615c-4389-8317-655ed0ebd2f3 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:54.761112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4a351-615c-4389-8317-655ed0ebd2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:54.761112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4a351-615c-4389-8317-655ed0ebd2f3 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:54.762779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4a351-615c-4389-8317-655ed0ebd2f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:54.763308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4a351-615c-4389-8317-655ed0ebd2f3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:54.872066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 439/881] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:30:54 2022] POST /identity/v3/services => generated 269 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:30:55.103031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363a348-2dfd-4c52-ae58-6d333ec90fa9 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:55.104144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363a348-2dfd-4c52-ae58-6d333ec90fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:55.104144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363a348-2dfd-4c52-ae58-6d333ec90fa9 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:55.105352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 443/882] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:55.111229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29b2e7-f852-482e-8b7a-227327b40424 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:55.111521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29b2e7-f852-482e-8b7a-227327b40424 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:55.111781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29b2e7-f852-482e-8b7a-227327b40424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:55.131055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ae29b2e7-f852-482e-8b7a-227327b40424 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:55.132230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ae29b2e7-f852-482e-8b7a-227327b40424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:55.218826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 440/883] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:55.485488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363a348-2dfd-4c52-ae58-6d333ec90fa9 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:55.596606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363a348-2dfd-4c52-ae58-6d333ec90fa9 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:55.604269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:55.631866 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb39eb-bdef-44c1-966b-167f493bc0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:55.632585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb39eb-bdef-44c1-966b-167f493bc0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:55.632859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb39eb-bdef-44c1-966b-167f493bc0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:55.633132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb39eb-bdef-44c1-966b-167f493bc0c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:55.644956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb39eb-bdef-44c1-966b-167f493bc0c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-dcbb39eb-bdef-44c1-966b-167f493bc0c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:30:55.651369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:55.652423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:55.653974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 444/884] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Sep 17 03:30:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:55.658721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29b2e7-f852-482e-8b7a-227327b40424 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:55.765781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29b2e7-f852-482e-8b7a-227327b40424 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:55.797164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:55.809065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2891274-adb5-4b6a-8f28-557e833e7962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:55.809688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2891274-adb5-4b6a-8f28-557e833e7962 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:55.810611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2891274-adb5-4b6a-8f28-557e833e7962 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:55.810611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2891274-adb5-4b6a-8f28-557e833e7962 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:55.810941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2891274-adb5-4b6a-8f28-557e833e7962 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:55.811309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2891274-adb5-4b6a-8f28-557e833e7962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:55.812578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2891274-adb5-4b6a-8f28-557e833e7962 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:55.821946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 441/885] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Sep 17 03:30:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:55.827619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb39eb-bdef-44c1-966b-167f493bc0c8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:55.936792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb39eb-bdef-44c1-966b-167f493bc0c8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:55.970989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07146fc-0267-461c-8770-4c07b4cb262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:55.971611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07146fc-0267-461c-8770-4c07b4cb262e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:55.971884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07146fc-0267-461c-8770-4c07b4cb262e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:55.972152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07146fc-0267-461c-8770-4c07b4cb262e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:55.984043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07146fc-0267-461c-8770-4c07b4cb262e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:55.986482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07146fc-0267-461c-8770-4c07b4cb262e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-f07146fc-0267-461c-8770-4c07b4cb262e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:55.999856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:56.000946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:56.002483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:56.002483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:56.002483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:56.002483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 445/886] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:56.006854 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2891274-adb5-4b6a-8f28-557e833e7962 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:56.109385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2891274-adb5-4b6a-8f28-557e833e7962 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:56.143080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c3f9f-0747-4ddc-a6c0-e201c00f11ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:56.143680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c3f9f-0747-4ddc-a6c0-e201c00f11ba None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:56.143956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c3f9f-0747-4ddc-a6c0-e201c00f11ba None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:56.144229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c3f9f-0747-4ddc-a6c0-e201c00f11ba None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:56.144865 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c3f9f-0747-4ddc-a6c0-e201c00f11ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:56.145166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c3f9f-0747-4ddc-a6c0-e201c00f11ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:56.146539 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c3f9f-0747-4ddc-a6c0-e201c00f11ba None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:56.159154 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 442/887] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:56.164620 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07146fc-0267-461c-8770-4c07b4cb262e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:56.190491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:56.281097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07146fc-0267-461c-8770-4c07b4cb262e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:56.323142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60a172-738c-4a5d-b8a1-375c83438370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:56.324428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60a172-738c-4a5d-b8a1-375c83438370 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:56.324919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60a172-738c-4a5d-b8a1-375c83438370 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:56.325408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60a172-738c-4a5d-b8a1-375c83438370 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:56.333915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60a172-738c-4a5d-b8a1-375c83438370 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:56.335598 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60a172-738c-4a5d-b8a1-375c83438370 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-5b60a172-738c-4a5d-b8a1-375c83438370 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:30:56.347229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:56.348217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:56.349243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:56.349243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:56.349243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:30:56.349243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:30:56.349243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:30:56.349243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:30:56.349243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:30:56.349243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:30:56.349243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:30:56.349243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:56.351865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 446/888] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Sep 17 03:30:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:30:56.357975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c3f9f-0747-4ddc-a6c0-e201c00f11ba None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:56.410122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:56.466287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c3f9f-0747-4ddc-a6c0-e201c00f11ba None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:56.499698 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a17d-fd4e-41d2-9e58-caa1796e8f64 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:56.500249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a17d-fd4e-41d2-9e58-caa1796e8f64 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:56.500838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a17d-fd4e-41d2-9e58-caa1796e8f64 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:56.501178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922d255-1410-42fa-bdc6-b055b6c40557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:56.501810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d255-1410-42fa-bdc6-b055b6c40557 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:56.502079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d255-1410-42fa-bdc6-b055b6c40557 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:56.502354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d255-1410-42fa-bdc6-b055b6c40557 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:56.503077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922d255-1410-42fa-bdc6-b055b6c40557 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:56.503380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922d255-1410-42fa-bdc6-b055b6c40557 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:56.504098 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 447/889] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:56.509896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922d255-1410-42fa-bdc6-b055b6c40557 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:56.510055 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d9e0c-c461-4f98-b457-29747b170bc1 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:56.510268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d9e0c-c461-4f98-b457-29747b170bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:56.512056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d9e0c-c461-4f98-b457-29747b170bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:56.523599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 443/890] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Sep 17 03:30:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:56.528786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922d255-1410-42fa-bdc6-b055b6c40557 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:56.529338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-9e4d9e0c-c461-4f98-b457-29747b170bc1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:56.530558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9e4d9e0c-c461-4f98-b457-29747b170bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:56.613286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 448/891] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:56.625830 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:56.631776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922d255-1410-42fa-bdc6-b055b6c40557 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:56.667618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5f8d5-d291-4f57-bda1-affc06f64934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:56.668265 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5f8d5-d291-4f57-bda1-affc06f64934 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:56.668535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5f8d5-d291-4f57-bda1-affc06f64934 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:56.668713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5f8d5-d291-4f57-bda1-affc06f64934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:56.739246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5f8d5-d291-4f57-bda1-affc06f64934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:56.740693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5f8d5-d291-4f57-bda1-affc06f64934 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:56.849103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 444/892] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:30:56 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:30:56.853744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:56.855317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d9e0c-c461-4f98-b457-29747b170bc1 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:56.866016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:56.880844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cd567-068b-4a5a-bf91-f00a4aea7e09 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:56.881423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cd567-068b-4a5a-bf91-f00a4aea7e09 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:56.881697 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cd567-068b-4a5a-bf91-f00a4aea7e09 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:56.883306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 445/893] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:56.891355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daae41f-bd07-4799-8aa5-30903e615d52 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:56.899717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daae41f-bd07-4799-8aa5-30903e615d52 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:56.899717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daae41f-bd07-4799-8aa5-30903e615d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:56.912488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4daae41f-bd07-4799-8aa5-30903e615d52 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:56.914033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4daae41f-bd07-4799-8aa5-30903e615d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:56.978086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d9e0c-c461-4f98-b457-29747b170bc1 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:57.009750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 446/894] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:30:57.009750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daae41f-bd07-4799-8aa5-30903e615d52 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:57.022522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bd68d-6a86-43cd-b7f8-bbad11736848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:57.023158 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd68d-6a86-43cd-b7f8-bbad11736848 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.023434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd68d-6a86-43cd-b7f8-bbad11736848 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.023705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd68d-6a86-43cd-b7f8-bbad11736848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.039531 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bd68d-6a86-43cd-b7f8-bbad11736848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:57.043253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bd68d-6a86-43cd-b7f8-bbad11736848 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-831bd68d-6a86-43cd-b7f8-bbad11736848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:30:57.062189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:57.063183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:30:57.064119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:30:57.064119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:30:57.064119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:57.066091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 449/895] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:57.071924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bd68d-6a86-43cd-b7f8-bbad11736848 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:57.135364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daae41f-bd07-4799-8aa5-30903e615d52 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:57.156592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:57.169043 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:57.179550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ad672-b847-4a1c-bde7-2b337bbfefcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:57.180580 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ad672-b847-4a1c-bde7-2b337bbfefcb None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.180860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ad672-b847-4a1c-bde7-2b337bbfefcb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.181190 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ad672-b847-4a1c-bde7-2b337bbfefcb None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.181906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ad672-b847-4a1c-bde7-2b337bbfefcb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:57.184109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ad672-b847-4a1c-bde7-2b337bbfefcb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:57.207431 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bd68d-6a86-43cd-b7f8-bbad11736848 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:57.248066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef83eaa-0fd8-48cd-bbf9-ace74e3081c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:57.248591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef83eaa-0fd8-48cd-bbf9-ace74e3081c4 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.248869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef83eaa-0fd8-48cd-bbf9-ace74e3081c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.249140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef83eaa-0fd8-48cd-bbf9-ace74e3081c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.249814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef83eaa-0fd8-48cd-bbf9-ace74e3081c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:57.250115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef83eaa-0fd8-48cd-bbf9-ace74e3081c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:57.251509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef83eaa-0fd8-48cd-bbf9-ace74e3081c4 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:57.269151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 450/896] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:57.269151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547d094-8226-4966-8993-4fc37d54d3d9 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.269151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547d094-8226-4966-8993-4fc37d54d3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.269151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547d094-8226-4966-8993-4fc37d54d3d9 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.269151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 451/897] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:57.270007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547d094-8226-4966-8993-4fc37d54d3d9 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:57.277070 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 447/898] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:30:56 2022] POST /identity/v3/services => generated 241 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:30:57.282105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae790578-c082-4770-b4a0-c72544f02b3e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.282105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae790578-c082-4770-b4a0-c72544f02b3e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.282105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae790578-c082-4770-b4a0-c72544f02b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.305337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ae790578-c082-4770-b4a0-c72544f02b3e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:57.307695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ae790578-c082-4770-b4a0-c72544f02b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:57.398724 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 448/899] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:57.399774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547d094-8226-4966-8993-4fc37d54d3d9 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:57.400296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae790578-c082-4770-b4a0-c72544f02b3e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:57.435771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:57.457391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e440f-f68a-4b4c-8f12-075467e71e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:57.458378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e440f-f68a-4b4c-8f12-075467e71e94 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.458875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e440f-f68a-4b4c-8f12-075467e71e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.459366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e440f-f68a-4b4c-8f12-075467e71e94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.469144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e440f-f68a-4b4c-8f12-075467e71e94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:57.470722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e440f-f68a-4b4c-8f12-075467e71e94 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-7e5e440f-f68a-4b4c-8f12-075467e71e94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:30:57.483855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:57.484851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:57.486156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:30:57.486156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:30:57.486156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:30:57.486156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:30:57.486156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:30:57.486156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:30:57.486156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:30:57.486156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:57.486156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 452/900] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:57.490959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e440f-f68a-4b4c-8f12-075467e71e94 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:57.537282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae790578-c082-4770-b4a0-c72544f02b3e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:57.562102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:57.594352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35631e-51e6-4864-9ae1-412394c6e3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:57.595392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35631e-51e6-4864-9ae1-412394c6e3ad None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.595392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35631e-51e6-4864-9ae1-412394c6e3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.595815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35631e-51e6-4864-9ae1-412394c6e3ad None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.596736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35631e-51e6-4864-9ae1-412394c6e3ad None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:57.598546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35631e-51e6-4864-9ae1-412394c6e3ad None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-fe35631e-51e6-4864-9ae1-412394c6e3ad None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:57.608138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:57.609444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 03:30:57.610413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:30:57.611186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 449/901] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:30:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:57.616729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35631e-51e6-4864-9ae1-412394c6e3ad None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:57.634611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e440f-f68a-4b4c-8f12-075467e71e94 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:57.677793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d089c-2d52-4310-b92a-c1e1fd4805b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:57.678495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d089c-2d52-4310-b92a-c1e1fd4805b7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.678868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d089c-2d52-4310-b92a-c1e1fd4805b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.679207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d089c-2d52-4310-b92a-c1e1fd4805b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.679958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d089c-2d52-4310-b92a-c1e1fd4805b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:57.680904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d089c-2d52-4310-b92a-c1e1fd4805b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:57.682558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d089c-2d52-4310-b92a-c1e1fd4805b7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:57.694506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 453/902] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:30:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:57.695471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d089c-2d52-4310-b92a-c1e1fd4805b7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:57.735300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:57.740399 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35631e-51e6-4864-9ae1-412394c6e3ad None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:57.782911 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fa0d2-e5bf-4b94-bf40-4a6d8c78d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:57.783515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fa0d2-e5bf-4b94-bf40-4a6d8c78d29e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.783793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fa0d2-e5bf-4b94-bf40-4a6d8c78d29e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.784217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fa0d2-e5bf-4b94-bf40-4a6d8c78d29e None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.784957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fa0d2-e5bf-4b94-bf40-4a6d8c78d29e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:57.785264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fa0d2-e5bf-4b94-bf40-4a6d8c78d29e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:57.786661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fa0d2-e5bf-4b94-bf40-4a6d8c78d29e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:57.792057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 450/903] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:30:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:30:57.792623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fa0d2-e5bf-4b94-bf40-4a6d8c78d29e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:57.814455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d089c-2d52-4310-b92a-c1e1fd4805b7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:57.840843 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:57.850525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95abc6c-9c05-46e0-86ee-4390e3dde655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:57.851159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95abc6c-9c05-46e0-86ee-4390e3dde655 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.853450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95abc6c-9c05-46e0-86ee-4390e3dde655 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.853450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95abc6c-9c05-46e0-86ee-4390e3dde655 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.853450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95abc6c-9c05-46e0-86ee-4390e3dde655 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:57.854365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95abc6c-9c05-46e0-86ee-4390e3dde655 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-c95abc6c-9c05-46e0-86ee-4390e3dde655 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:30:57.870940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:30:57.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:30:57.872923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 454/904] 10.222.0.42 () {56 vars in 1132 bytes} [Sat Sep 17 03:30:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:30:57.876418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95abc6c-9c05-46e0-86ee-4390e3dde655 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:57.915292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fa0d2-e5bf-4b94-bf40-4a6d8c78d29e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:57.949565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a32d2-9dd9-4f09-835c-648e05b1b105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:57.950198 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a32d2-9dd9-4f09-835c-648e05b1b105 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:57.950484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a32d2-9dd9-4f09-835c-648e05b1b105 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:57.950760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a32d2-9dd9-4f09-835c-648e05b1b105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:57.951482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a32d2-9dd9-4f09-835c-648e05b1b105 None admin] RBAC: Adding query filter params (role.id=e88868b1e3fc44e6aee8a63eb09ddd9b, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=b3803e9daa1848ac9feec1bdef4130de) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:57.951794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a32d2-9dd9-4f09-835c-648e05b1b105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:57.953240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a32d2-9dd9-4f09-835c-648e05b1b105 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:57.958764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 451/905] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:30:57 2022] GET /identity/v3/role_assignments?user.id=b3803e9daa1848ac9feec1bdef4130de&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e88868b1e3fc44e6aee8a63eb09ddd9b => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:57.970802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a32d2-9dd9-4f09-835c-648e05b1b105 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:57.979245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:57.984954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95abc6c-9c05-46e0-86ee-4390e3dde655 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:58.031325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef3be3-fa9a-4261-8543-4c2d4eb13bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:58.032147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef3be3-fa9a-4261-8543-4c2d4eb13bfa None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:58.032562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef3be3-fa9a-4261-8543-4c2d4eb13bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:58.032967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef3be3-fa9a-4261-8543-4c2d4eb13bfa None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:58.033715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef3be3-fa9a-4261-8543-4c2d4eb13bfa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:58.034247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef3be3-fa9a-4261-8543-4c2d4eb13bfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:58.035933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef3be3-fa9a-4261-8543-4c2d4eb13bfa None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:58.047285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 455/906] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:30:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:58.048424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef3be3-fa9a-4261-8543-4c2d4eb13bfa None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:58.100076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a32d2-9dd9-4f09-835c-648e05b1b105 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:58.103501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:58.141694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab6005-9e2e-4a58-827a-f7868cf14ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:58.142498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab6005-9e2e-4a58-827a-f7868cf14ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:58.142779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab6005-9e2e-4a58-827a-f7868cf14ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:58.143433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab6005-9e2e-4a58-827a-f7868cf14ee0 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:58.143696 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab6005-9e2e-4a58-827a-f7868cf14ee0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:58.143997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab6005-9e2e-4a58-827a-f7868cf14ee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:58.145373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab6005-9e2e-4a58-827a-f7868cf14ee0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:58.151982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 452/907] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:30:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:58.158486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab6005-9e2e-4a58-827a-f7868cf14ee0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:58.176690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef3be3-fa9a-4261-8543-4c2d4eb13bfa None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:58.215670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce9269-b13a-4007-bac2-f45a01d852d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:58.217200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce9269-b13a-4007-bac2-f45a01d852d9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:58.217697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce9269-b13a-4007-bac2-f45a01d852d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:58.218237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce9269-b13a-4007-bac2-f45a01d852d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:58.219139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce9269-b13a-4007-bac2-f45a01d852d9 None admin] RBAC: Adding query filter params (interface=public, service_id=98d93f65e0c54952897728bfde65b6ac, region_id=RegionOne) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:58.219650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce9269-b13a-4007-bac2-f45a01d852d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:58.221437 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce9269-b13a-4007-bac2-f45a01d852d9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:58.228463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 456/908] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Sep 17 03:30:58 2022] GET /identity/v3/endpoints?service_id=98d93f65e0c54952897728bfde65b6ac&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:58.236976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce9269-b13a-4007-bac2-f45a01d852d9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:58.268659 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab6005-9e2e-4a58-827a-f7868cf14ee0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:58.271500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:58.306513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01b67c-67b6-4294-adf4-e9cfaa3d2c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:58.307043 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01b67c-67b6-4294-adf4-e9cfaa3d2c5e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:58.307828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01b67c-67b6-4294-adf4-e9cfaa3d2c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:58.307828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01b67c-67b6-4294-adf4-e9cfaa3d2c5e None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:58.311677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01b67c-67b6-4294-adf4-e9cfaa3d2c5e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:58.311677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01b67c-67b6-4294-adf4-e9cfaa3d2c5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:58.312347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01b67c-67b6-4294-adf4-e9cfaa3d2c5e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:58.318070 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 453/909] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:30:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:58.327333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01b67c-67b6-4294-adf4-e9cfaa3d2c5e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:58.363501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce9269-b13a-4007-bac2-f45a01d852d9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:58.375273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:58.401992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d9e7a-56b1-4172-89be-cf5526a7f981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:58.402601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9e7a-56b1-4172-89be-cf5526a7f981 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:58.402879 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9e7a-56b1-4172-89be-cf5526a7f981 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:58.403152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9e7a-56b1-4172-89be-cf5526a7f981 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:58.404041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d9e7a-56b1-4172-89be-cf5526a7f981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:58.405519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d9e7a-56b1-4172-89be-cf5526a7f981 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:58.417778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 457/910] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Sep 17 03:30:58 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:30:58.476957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01b67c-67b6-4294-adf4-e9cfaa3d2c5e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:58.517050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61246b-adca-46e6-8b93-871e19e1855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:58.517759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61246b-adca-46e6-8b93-871e19e1855b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:58.518056 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61246b-adca-46e6-8b93-871e19e1855b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:58.518335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61246b-adca-46e6-8b93-871e19e1855b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:58.519094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61246b-adca-46e6-8b93-871e19e1855b None admin] RBAC: Adding query filter params (interface=public, service_id=9de5ab36c0074594ab2f3d5f7be69079, region_id=RegionOne) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:30:58.519410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61246b-adca-46e6-8b93-871e19e1855b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:58.520830 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61246b-adca-46e6-8b93-871e19e1855b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:58.532906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 454/911] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Sep 17 03:30:58 2022] GET /identity/v3/endpoints?service_id=9de5ab36c0074594ab2f3d5f7be69079&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:30:58.534497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d9e7a-56b1-4172-89be-cf5526a7f981 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:30:58.637704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:30:58.650726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d9e7a-56b1-4172-89be-cf5526a7f981 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:30:58.688255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a16e80-605a-4125-be95-9b32ff42dba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:30:58.688849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a16e80-605a-4125-be95-9b32ff42dba5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:58.689118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a16e80-605a-4125-be95-9b32ff42dba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:58.689385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a16e80-605a-4125-be95-9b32ff42dba5 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:58.690010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a16e80-605a-4125-be95-9b32ff42dba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:30:58.691368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a16e80-605a-4125-be95-9b32ff42dba5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:30:58.698320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 458/912] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Sep 17 03:30:58 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:30:59.569671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555d744-19cd-4551-b243-bbe132824cf7 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:59.570215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555d744-19cd-4551-b243-bbe132824cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:59.570293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555d744-19cd-4551-b243-bbe132824cf7 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:59.572520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 455/913] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:30:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:30:59.581252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ac6ad-5c7f-4d6b-8c0c-1bcc27dafc0d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:30:59.581252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ac6ad-5c7f-4d6b-8c0c-1bcc27dafc0d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:30:59.581252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ac6ad-5c7f-4d6b-8c0c-1bcc27dafc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:30:59.598456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-bb5ac6ad-5c7f-4d6b-8c0c-1bcc27dafc0d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:30:59.600019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bb5ac6ad-5c7f-4d6b-8c0c-1bcc27dafc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:30:59.685564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 459/914] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:30:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:30:59.956206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555d744-19cd-4551-b243-bbe132824cf7 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:00.058187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:00.065674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555d744-19cd-4551-b243-bbe132824cf7 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:00.102003 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94781b0d-bc01-4ea3-a864-d3291e9a1a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:00.103041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94781b0d-bc01-4ea3-a864-d3291e9a1a6d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.103547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94781b0d-bc01-4ea3-a864-d3291e9a1a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.104027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94781b0d-bc01-4ea3-a864-d3291e9a1a6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.104837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94781b0d-bc01-4ea3-a864-d3291e9a1a6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:00.106440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94781b0d-bc01-4ea3-a864-d3291e9a1a6d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-94781b0d-bc01-4ea3-a864-d3291e9a1a6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:00.116359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:00.117380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:00.118454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:00.121247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 456/915] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:30:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:00.128571 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ac6ad-5c7f-4d6b-8c0c-1bcc27dafc0d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:00.230629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:00.247950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ac6ad-5c7f-4d6b-8c0c-1bcc27dafc0d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:00.284778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a1824-80a6-4ba4-92d5-cb652227ba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:00.285381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a1824-80a6-4ba4-92d5-cb652227ba65 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.285655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a1824-80a6-4ba4-92d5-cb652227ba65 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.289087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a1824-80a6-4ba4-92d5-cb652227ba65 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.289087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a1824-80a6-4ba4-92d5-cb652227ba65 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:00.289087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a1824-80a6-4ba4-92d5-cb652227ba65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:00.289087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a1824-80a6-4ba4-92d5-cb652227ba65 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:00.293855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 460/916] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Sep 17 03:31:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:00.299579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94781b0d-bc01-4ea3-a864-d3291e9a1a6d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:00.311909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff82741-ce4e-4034-883b-efab83d95f64 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.312203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff82741-ce4e-4034-883b-efab83d95f64 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.312472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff82741-ce4e-4034-883b-efab83d95f64 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.314143 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 461/917] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:31:00.319132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693246-f59e-436f-8b75-708176e43c68 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.319415 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693246-f59e-436f-8b75-708176e43c68 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.319691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693246-f59e-436f-8b75-708176e43c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.339200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-19693246-f59e-436f-8b75-708176e43c68 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:00.340398 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-19693246-f59e-436f-8b75-708176e43c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:00.421226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94781b0d-bc01-4ea3-a864-d3291e9a1a6d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:00.421850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 462/918] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:00.459396 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a0488-5cba-4afc-a41e-6037431e87e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:00.460357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a0488-5cba-4afc-a41e-6037431e87e1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.460836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a0488-5cba-4afc-a41e-6037431e87e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.461306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a0488-5cba-4afc-a41e-6037431e87e1 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.463078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a0488-5cba-4afc-a41e-6037431e87e1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:00.463670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a0488-5cba-4afc-a41e-6037431e87e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:00.465222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a0488-5cba-4afc-a41e-6037431e87e1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:00.471125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 457/919] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Sep 17 03:31:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:00.478591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19693246-f59e-436f-8b75-708176e43c68 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:00.574425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:00.588682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19693246-f59e-436f-8b75-708176e43c68 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:00.628653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d3287-3a79-462d-b41b-073713592c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:00.629302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d3287-3a79-462d-b41b-073713592c8e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.629591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d3287-3a79-462d-b41b-073713592c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.629895 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d3287-3a79-462d-b41b-073713592c8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.630608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d3287-3a79-462d-b41b-073713592c8e None admin] RBAC: Adding query filter params (interface=public, service_id=973838b71feb4dc8a27d4d1654567523, region_id=RegionOne) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:00.630926 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d3287-3a79-462d-b41b-073713592c8e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:00.632319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d3287-3a79-462d-b41b-073713592c8e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:00.638484 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 463/920] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Sep 17 03:31:00 2022] GET /identity/v3/endpoints?service_id=973838b71feb4dc8a27d4d1654567523&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:00.645832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a0488-5cba-4afc-a41e-6037431e87e1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:00.675829 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:00.705770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1370c-b733-4b94-929b-642bda50f9c3 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.705770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1370c-b733-4b94-929b-642bda50f9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.705770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1370c-b733-4b94-929b-642bda50f9c3 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.708921 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 464/921] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:31:00.717054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b948e9-c131-49f0-9e1e-a5356888fbd5 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.717054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b948e9-c131-49f0-9e1e-a5356888fbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.717054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b948e9-c131-49f0-9e1e-a5356888fbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.733853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a9b948e9-c131-49f0-9e1e-a5356888fbd5 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:00.735028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a9b948e9-c131-49f0-9e1e-a5356888fbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:00.766965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a0488-5cba-4afc-a41e-6037431e87e1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:00.807808 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea77c9-74cf-4078-834b-ce8ca52f94d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:00.808380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea77c9-74cf-4078-834b-ce8ca52f94d2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.808635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea77c9-74cf-4078-834b-ce8ca52f94d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.809812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea77c9-74cf-4078-834b-ce8ca52f94d2 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.809812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea77c9-74cf-4078-834b-ce8ca52f94d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:00.818114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea77c9-74cf-4078-834b-ce8ca52f94d2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:00.818269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 465/922] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:00.818269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b948e9-c131-49f0-9e1e-a5356888fbd5 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:00.832035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 458/923] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Sep 17 03:31:00 2022] GET /identity/v3/services => generated 1497 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:31:00.907206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:00.932656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b948e9-c131-49f0-9e1e-a5356888fbd5 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:00.970534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bd07a-0262-4a97-ac7b-afc4dc67c2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:00.971186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bd07a-0262-4a97-ac7b-afc4dc67c2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.971467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bd07a-0262-4a97-ac7b-afc4dc67c2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.971742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bd07a-0262-4a97-ac7b-afc4dc67c2c7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.984833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bd07a-0262-4a97-ac7b-afc4dc67c2c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:00.985823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed33c7-d440-43eb-894d-d1e5630bde9c None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.986333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed33c7-d440-43eb-894d-d1e5630bde9c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.986824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed33c7-d440-43eb-894d-d1e5630bde9c None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.988689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 459/924] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-8b1bd07a-0262-4a97-ac7b-afc4dc67c2c7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:00.992114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:00.993517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:00.994504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 466/925] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Sep 17 03:31:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:00.997227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38c30-9d06-452c-8eae-9ce4c48833ee None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:00.997849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38c30-9d06-452c-8eae-9ce4c48833ee None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:00.998226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38c30-9d06-452c-8eae-9ce4c48833ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:00.999169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bd07a-0262-4a97-ac7b-afc4dc67c2c7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.019206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-aaa38c30-9d06-452c-8eae-9ce4c48833ee None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:01.020680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-aaa38c30-9d06-452c-8eae-9ce4c48833ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:01.109072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 460/926] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:01.109990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa38c30-9d06-452c-8eae-9ce4c48833ee None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.124928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bd07a-0262-4a97-ac7b-afc4dc67c2c7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:01.130323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:01.172480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1958e-dcf4-4dce-9729-3148e47e2f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:01.173138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1958e-dcf4-4dce-9729-3148e47e2f1d None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:01.173413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1958e-dcf4-4dce-9729-3148e47e2f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:01.173751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1958e-dcf4-4dce-9729-3148e47e2f1d None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:01.174422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1958e-dcf4-4dce-9729-3148e47e2f1d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:01.174721 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1958e-dcf4-4dce-9729-3148e47e2f1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:01.176963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1958e-dcf4-4dce-9729-3148e47e2f1d None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:01.179080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:01.190337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 467/927] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Sep 17 03:31:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:01.195785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1958e-dcf4-4dce-9729-3148e47e2f1d None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.238808 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa38c30-9d06-452c-8eae-9ce4c48833ee None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:01.285204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23086c1-202a-45ad-b3bc-d736ad73a238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:01.285922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23086c1-202a-45ad-b3bc-d736ad73a238 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:01.286210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23086c1-202a-45ad-b3bc-d736ad73a238 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:01.286538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23086c1-202a-45ad-b3bc-d736ad73a238 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:01.287193 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23086c1-202a-45ad-b3bc-d736ad73a238 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:01.288648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23086c1-202a-45ad-b3bc-d736ad73a238 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c23086c1-202a-45ad-b3bc-d736ad73a238 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:01.298507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:01.299526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:01.300391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 461/928] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:31:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:01.305822 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23086c1-202a-45ad-b3bc-d736ad73a238 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.315208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1958e-dcf4-4dce-9729-3148e47e2f1d None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:01.353036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98503-73a4-41db-9559-95c7d140a2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:01.354094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98503-73a4-41db-9559-95c7d140a2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:01.354848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98503-73a4-41db-9559-95c7d140a2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:01.355332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98503-73a4-41db-9559-95c7d140a2e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:01.372431 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e98503-73a4-41db-9559-95c7d140a2e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:01.374478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e98503-73a4-41db-9559-95c7d140a2e5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d4e98503-73a4-41db-9559-95c7d140a2e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:01.390291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:01.391275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:01.392178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:01.392178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:01.392178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:01.398542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 468/929] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:01.400424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98503-73a4-41db-9559-95c7d140a2e5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.412973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:01.432430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23086c1-202a-45ad-b3bc-d736ad73a238 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:01.456014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:01.470566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28119f77-e146-4329-9605-f2285154b4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:01.471175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28119f77-e146-4329-9605-f2285154b4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:01.471458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28119f77-e146-4329-9605-f2285154b4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:01.472383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28119f77-e146-4329-9605-f2285154b4e6 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:01.472383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28119f77-e146-4329-9605-f2285154b4e6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:01.472633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28119f77-e146-4329-9605-f2285154b4e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:01.474022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28119f77-e146-4329-9605-f2285154b4e6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:01.479935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 462/930] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:31:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:01.482767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28119f77-e146-4329-9605-f2285154b4e6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.518854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98503-73a4-41db-9559-95c7d140a2e5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:01.554033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22704ed-7539-43f0-af8e-d5b75a0801ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:01.554658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22704ed-7539-43f0-af8e-d5b75a0801ec None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:01.554940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22704ed-7539-43f0-af8e-d5b75a0801ec None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:01.555748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22704ed-7539-43f0-af8e-d5b75a0801ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:01.556341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22704ed-7539-43f0-af8e-d5b75a0801ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:01.557822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22704ed-7539-43f0-af8e-d5b75a0801ec None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-e22704ed-7539-43f0-af8e-d5b75a0801ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:01.566739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:01.567752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:01.568684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 469/931] 10.222.0.42 () {56 vars in 1132 bytes} [Sat Sep 17 03:31:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:01.569495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22704ed-7539-43f0-af8e-d5b75a0801ec None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.604512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28119f77-e146-4329-9605-f2285154b4e6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:01.654275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185210db-fd2e-48a1-a911-e7da71152a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:01.654920 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185210db-fd2e-48a1-a911-e7da71152a4d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:01.655153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185210db-fd2e-48a1-a911-e7da71152a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:01.655427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185210db-fd2e-48a1-a911-e7da71152a4d None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:01.656075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185210db-fd2e-48a1-a911-e7da71152a4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:01.656379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185210db-fd2e-48a1-a911-e7da71152a4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:01.657997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185210db-fd2e-48a1-a911-e7da71152a4d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:01.670520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 463/932] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:01.671564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185210db-fd2e-48a1-a911-e7da71152a4d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.683688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:01.698478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22704ed-7539-43f0-af8e-d5b75a0801ec None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:01.711392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:01.733635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a198a7a-f0ec-405d-a13d-2772e21c0a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:01.734540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a198a7a-f0ec-405d-a13d-2772e21c0a55 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:01.734816 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a198a7a-f0ec-405d-a13d-2772e21c0a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:01.735081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a198a7a-f0ec-405d-a13d-2772e21c0a55 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:01.735702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a198a7a-f0ec-405d-a13d-2772e21c0a55 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:01.735999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a198a7a-f0ec-405d-a13d-2772e21c0a55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:01.737787 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a198a7a-f0ec-405d-a13d-2772e21c0a55 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:01.747280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 470/933] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Sep 17 03:31:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:01.747280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a198a7a-f0ec-405d-a13d-2772e21c0a55 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.803858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185210db-fd2e-48a1-a911-e7da71152a4d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:01.838980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe19c-110a-449d-9b70-257ada1c67c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:01.839736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe19c-110a-449d-9b70-257ada1c67c4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:01.840023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe19c-110a-449d-9b70-257ada1c67c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:01.840656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe19c-110a-449d-9b70-257ada1c67c4 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:01.841292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fe19c-110a-449d-9b70-257ada1c67c4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:01.841598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fe19c-110a-449d-9b70-257ada1c67c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:01.842995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fe19c-110a-449d-9b70-257ada1c67c4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:01.852730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 464/934] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:31:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:01.852730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe19c-110a-449d-9b70-257ada1c67c4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.864523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a198a7a-f0ec-405d-a13d-2772e21c0a55 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:01.903474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd49de9-1228-4852-8f28-b225a97ae592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:01.904103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd49de9-1228-4852-8f28-b225a97ae592 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:01.904583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd49de9-1228-4852-8f28-b225a97ae592 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:01.904692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd49de9-1228-4852-8f28-b225a97ae592 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:01.912596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd49de9-1228-4852-8f28-b225a97ae592 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:01.914235 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd49de9-1228-4852-8f28-b225a97ae592 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-2dd49de9-1228-4852-8f28-b225a97ae592 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:31:01.926101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:01.927136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:01.928113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 471/935] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Sep 17 03:31:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:01.928718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd49de9-1228-4852-8f28-b225a97ae592 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:01.940290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:01.972057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:01.979412 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe19c-110a-449d-9b70-257ada1c67c4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:02.017339 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1966d-171f-4a57-8930-434743041f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:02.018018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1966d-171f-4a57-8930-434743041f9a None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:02.018295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1966d-171f-4a57-8930-434743041f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:02.018570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1966d-171f-4a57-8930-434743041f9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:02.019606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1966d-171f-4a57-8930-434743041f9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:02.021306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1966d-171f-4a57-8930-434743041f9a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:02.042952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd49de9-1228-4852-8f28-b225a97ae592 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:02.075806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85937b-d168-4a83-93d9-d97057a3a7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:02.076407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85937b-d168-4a83-93d9-d97057a3a7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:02.076687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85937b-d168-4a83-93d9-d97057a3a7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:02.076964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85937b-d168-4a83-93d9-d97057a3a7f3 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:02.077590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85937b-d168-4a83-93d9-d97057a3a7f3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:02.077954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85937b-d168-4a83-93d9-d97057a3a7f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:02.079342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85937b-d168-4a83-93d9-d97057a3a7f3 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:02.084048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 472/936] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:31:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:02.084823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85937b-d168-4a83-93d9-d97057a3a7f3 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:02.110507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 465/937] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:31:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:02.115827 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1966d-171f-4a57-8930-434743041f9a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:02.204039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:02.207283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85937b-d168-4a83-93d9-d97057a3a7f3 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:02.248378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1966d-171f-4a57-8930-434743041f9a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:02.253154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d559b-bbcb-4278-b344-7a0118b3a57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:02.253802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d559b-bbcb-4278-b344-7a0118b3a57f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:02.254079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d559b-bbcb-4278-b344-7a0118b3a57f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:02.254348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d559b-bbcb-4278-b344-7a0118b3a57f None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:02.255086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d559b-bbcb-4278-b344-7a0118b3a57f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:02.255457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d559b-bbcb-4278-b344-7a0118b3a57f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:02.256979 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d559b-bbcb-4278-b344-7a0118b3a57f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:02.281421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:02.282763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 473/938] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Sep 17 03:31:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:02.288158 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d559b-bbcb-4278-b344-7a0118b3a57f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:02.305466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77797e57-db24-4329-8854-6981b57a21bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:02.306252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77797e57-db24-4329-8854-6981b57a21bb None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:02.306534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77797e57-db24-4329-8854-6981b57a21bb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:02.306805 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77797e57-db24-4329-8854-6981b57a21bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:02.307466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77797e57-db24-4329-8854-6981b57a21bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:02.308831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77797e57-db24-4329-8854-6981b57a21bb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:02.406688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d559b-bbcb-4278-b344-7a0118b3a57f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:02.443120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 466/939] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:31:02 2022] POST /identity/v3/endpoints => generated 347 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:02.467360 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dec9b6-1559-4b6e-87c8-9351c82c2b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:02.468827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dec9b6-1559-4b6e-87c8-9351c82c2b1e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:02.469370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dec9b6-1559-4b6e-87c8-9351c82c2b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:02.469861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dec9b6-1559-4b6e-87c8-9351c82c2b1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:02.484111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dec9b6-1559-4b6e-87c8-9351c82c2b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:02.486079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dec9b6-1559-4b6e-87c8-9351c82c2b1e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-41dec9b6-1559-4b6e-87c8-9351c82c2b1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:02.504770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:02.506041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:02.507099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:02.507099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:02.507099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:02.509780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 474/940] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:02.515042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77797e57-db24-4329-8854-6981b57a21bb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:02.639201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77797e57-db24-4329-8854-6981b57a21bb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:02.677261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d60403-a69e-46c2-a618-cc76f4b3a842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:02.681493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d60403-a69e-46c2-a618-cc76f4b3a842 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:02.681493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d60403-a69e-46c2-a618-cc76f4b3a842 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:02.681493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d60403-a69e-46c2-a618-cc76f4b3a842 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:02.681493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d60403-a69e-46c2-a618-cc76f4b3a842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:02.681493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d60403-a69e-46c2-a618-cc76f4b3a842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:02.682043 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d60403-a69e-46c2-a618-cc76f4b3a842 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:02.694905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 467/941] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:02.700855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dec9b6-1559-4b6e-87c8-9351c82c2b1e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:02.711105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:02.824583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dec9b6-1559-4b6e-87c8-9351c82c2b1e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:02.860075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5285a-ca5f-4c53-b51e-a7924eb08a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:02.860825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5285a-ca5f-4c53-b51e-a7924eb08a93 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:02.861154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5285a-ca5f-4c53-b51e-a7924eb08a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:02.861427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5285a-ca5f-4c53-b51e-a7924eb08a93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:02.869287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5285a-ca5f-4c53-b51e-a7924eb08a93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:02.870835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5285a-ca5f-4c53-b51e-a7924eb08a93 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-f0c5285a-ca5f-4c53-b51e-a7924eb08a93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:31:02.885751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:02.887029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:02.887952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:31:02.887952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:31:02.887952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:31:02.887952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:31:02.887952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:31:02.887952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:31:02.887952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:02.887952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:02.887952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 475/942] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:02.891364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d60403-a69e-46c2-a618-cc76f4b3a842 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:02.937151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:03.000688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d60403-a69e-46c2-a618-cc76f4b3a842 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:03.038642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d5b7e-9560-4766-8c2f-5ff0574ec4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:03.039249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d5b7e-9560-4766-8c2f-5ff0574ec4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:03.039625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d5b7e-9560-4766-8c2f-5ff0574ec4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:03.039902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d5b7e-9560-4766-8c2f-5ff0574ec4e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:03.040596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d5b7e-9560-4766-8c2f-5ff0574ec4e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:03.040894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d5b7e-9560-4766-8c2f-5ff0574ec4e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:03.042421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d5b7e-9560-4766-8c2f-5ff0574ec4e5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:03.056589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 468/943] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:03.062894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5285a-ca5f-4c53-b51e-a7924eb08a93 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:03.139297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:03.178731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5285a-ca5f-4c53-b51e-a7924eb08a93 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:03.183108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d625a-9266-4285-8c10-d126db38980e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:03.183659 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d625a-9266-4285-8c10-d126db38980e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:03.184164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d625a-9266-4285-8c10-d126db38980e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:03.185970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 469/944] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:03.191131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901eeb6-58f4-4933-b31c-b50aba4906d7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:03.191650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901eeb6-58f4-4933-b31c-b50aba4906d7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:03.192142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901eeb6-58f4-4933-b31c-b50aba4906d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:03.214157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c901eeb6-58f4-4933-b31c-b50aba4906d7 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:03.215822 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c901eeb6-58f4-4933-b31c-b50aba4906d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:03.220872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01b9db-9384-4b36-85c2-e5947fe195d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:03.222057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01b9db-9384-4b36-85c2-e5947fe195d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:03.222611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01b9db-9384-4b36-85c2-e5947fe195d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:03.223030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01b9db-9384-4b36-85c2-e5947fe195d5 None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/b3803e9daa1848ac9feec1bdef4130de/roles/e88868b1e3fc44e6aee8a63eb09ddd9b` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:03.226757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01b9db-9384-4b36-85c2-e5947fe195d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=b3803e9daa1848ac9feec1bdef4130de, role_id=e88868b1e3fc44e6aee8a63eb09ddd9b)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:03.229157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01b9db-9384-4b36-85c2-e5947fe195d5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:03.288301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 470/945] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:03.311275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 476/946] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:31:03 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/b3803e9daa1848ac9feec1bdef4130de/roles/e88868b1e3fc44e6aee8a63eb09ddd9b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:31:03.576545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901eeb6-58f4-4933-b31c-b50aba4906d7 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:03.616642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:03.697452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901eeb6-58f4-4933-b31c-b50aba4906d7 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:03.735028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bfcb8-c92b-4280-9011-c074e8491e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:03.737749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bfcb8-c92b-4280-9011-c074e8491e4b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:03.737749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bfcb8-c92b-4280-9011-c074e8491e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:03.737749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bfcb8-c92b-4280-9011-c074e8491e4b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:03.737749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bfcb8-c92b-4280-9011-c074e8491e4b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:03.738186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bfcb8-c92b-4280-9011-c074e8491e4b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e79bfcb8-c92b-4280-9011-c074e8491e4b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:03.749745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:03.750754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:03.751611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 471/947] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:31:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:03.755288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01b9db-9384-4b36-85c2-e5947fe195d5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:03.877951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01b9db-9384-4b36-85c2-e5947fe195d5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:03.906073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:03.915940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd484ae-cca7-4368-974a-9f44c422f11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:03.916572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd484ae-cca7-4368-974a-9f44c422f11f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:03.916856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd484ae-cca7-4368-974a-9f44c422f11f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:03.917121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd484ae-cca7-4368-974a-9f44c422f11f None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:03.923977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd484ae-cca7-4368-974a-9f44c422f11f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:03.923977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd484ae-cca7-4368-974a-9f44c422f11f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:03.923977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd484ae-cca7-4368-974a-9f44c422f11f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:03.924632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 477/948] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Sep 17 03:31:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:03.930815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bfcb8-c92b-4280-9011-c074e8491e4b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:04.037814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:04.048456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bfcb8-c92b-4280-9011-c074e8491e4b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:04.085312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10143b8-ddd6-4854-aaf4-6866f93cdb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:04.085952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10143b8-ddd6-4854-aaf4-6866f93cdb25 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:04.086234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10143b8-ddd6-4854-aaf4-6866f93cdb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:04.086598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10143b8-ddd6-4854-aaf4-6866f93cdb25 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:04.087232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10143b8-ddd6-4854-aaf4-6866f93cdb25 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:04.087537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10143b8-ddd6-4854-aaf4-6866f93cdb25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:04.088898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10143b8-ddd6-4854-aaf4-6866f93cdb25 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:04.094376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 472/949] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Sep 17 03:31:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:04.099697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd484ae-cca7-4368-974a-9f44c422f11f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:04.207855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd484ae-cca7-4368-974a-9f44c422f11f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:04.248394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f67218-a343-45ce-83c5-2e0bf7d32a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:04.248394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f67218-a343-45ce-83c5-2e0bf7d32a60 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:04.248394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f67218-a343-45ce-83c5-2e0bf7d32a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:04.248394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f67218-a343-45ce-83c5-2e0bf7d32a60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:04.249011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f67218-a343-45ce-83c5-2e0bf7d32a60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:04.253751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f67218-a343-45ce-83c5-2e0bf7d32a60 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:04.345540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 478/950] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:31:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:04.406231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447bab9-ba53-435e-b4e9-a7bd48254b3e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:04.406600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447bab9-ba53-435e-b4e9-a7bd48254b3e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:04.406795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447bab9-ba53-435e-b4e9-a7bd48254b3e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:04.408461 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 473/951] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:04.414301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d1f56-dcf4-4917-b09c-2be4cbde1e3e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:04.414697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d1f56-dcf4-4917-b09c-2be4cbde1e3e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:04.414984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d1f56-dcf4-4917-b09c-2be4cbde1e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:04.435814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-302d1f56-dcf4-4917-b09c-2be4cbde1e3e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:04.437122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-302d1f56-dcf4-4917-b09c-2be4cbde1e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:04.489184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:04.526612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 479/952] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:04.809079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447bab9-ba53-435e-b4e9-a7bd48254b3e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:04.849101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe9804-897f-40f7-8277-e1d22603085c None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:04.849627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe9804-897f-40f7-8277-e1d22603085c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:04.849627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe9804-897f-40f7-8277-e1d22603085c None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:04.852904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 480/953] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:04.857103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24763582-e756-4929-add2-321bffe46c36 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:04.857389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24763582-e756-4929-add2-321bffe46c36 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:04.857662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24763582-e756-4929-add2-321bffe46c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:04.879019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-24763582-e756-4929-add2-321bffe46c36 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:04.880686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-24763582-e756-4929-add2-321bffe46c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:04.927679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447bab9-ba53-435e-b4e9-a7bd48254b3e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:04.956829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 481/954] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:04.973749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:04.973749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:04.973749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:04.973749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:04.974152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:04.974152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:04.977746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WARNING keystone.server.flask.common [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 17 03:31:05.046232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.sql.core [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 03:31:05.046232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 03:31:05.046232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [parameters: {'id': '91f0a7b884b34d769b90a4000ac71ffa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 17 03:31:05.046232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81725) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 91f0a7b884b34d769b90a4000ac71ffa.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 91f0a7b884b34d769b90a4000ac71ffa. Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application conn.query(q) Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 03:31:05.053994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result.read() Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 17 03:31:05.054815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application next(self.gen) Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application yield resource Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application next(self.gen) Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise self.value Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application session.commit() Sep 17 03:31:05.056085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.session.flush() Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application compat.raise_( Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 03:31:05.061762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 03:31:05.062874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application conn.query(q) Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result.read() Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 03:31:05.063978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [parameters: {'id': '91f0a7b884b34d769b90a4000ac71ffa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:05.065083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:05.066145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:05.067256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:05.067256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 17 03:31:05.067256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 17 03:31:05.067256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:05.067256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:05.067256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 17 03:31:05.067256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 17 03:31:05.067256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 91f0a7b884b34d769b90a4000ac71ffa. Sep 17 03:31:05.067256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:05.067256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 474/955] 10.222.0.42 () {60 vars in 1140 bytes} [Sat Sep 17 03:31:04 2022] POST /identity/v3/projects => generated 262 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:05.067833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24763582-e756-4929-add2-321bffe46c36 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:05.175462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24763582-e756-4929-add2-321bffe46c36 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:05.177022 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:05.215638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65805f1-7e04-4d53-907d-c3f54f007925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:05.216267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65805f1-7e04-4d53-907d-c3f54f007925 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:05.216544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65805f1-7e04-4d53-907d-c3f54f007925 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:05.216853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65805f1-7e04-4d53-907d-c3f54f007925 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:05.223988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65805f1-7e04-4d53-907d-c3f54f007925 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:05.225447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65805f1-7e04-4d53-907d-c3f54f007925 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-e65805f1-7e04-4d53-907d-c3f54f007925 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:31:05.238155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:05.240175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:05.242048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:31:05.242048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:31:05.242048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:31:05.242048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:31:05.242048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:31:05.242048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:31:05.242048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:05.242048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:05.242048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 482/956] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:31:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:05.246882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:05.253040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65805f1-7e04-4d53-907d-c3f54f007925 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:05.263537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:05.375796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffc1b1-3d6a-4049-8deb-6ad1358ce237 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:05.378139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65805f1-7e04-4d53-907d-c3f54f007925 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:05.417648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ad85c-b59a-40b8-8f72-084acbd3c111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:05.418642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ad85c-b59a-40b8-8f72-084acbd3c111 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:05.419144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ad85c-b59a-40b8-8f72-084acbd3c111 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:05.421235 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ad85c-b59a-40b8-8f72-084acbd3c111 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:05.422085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ad85c-b59a-40b8-8f72-084acbd3c111 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:05.422356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc4166-61f2-4f9e-af45-8e98b7ae749e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:05.422964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc4166-61f2-4f9e-af45-8e98b7ae749e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:05.423242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc4166-61f2-4f9e-af45-8e98b7ae749e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:05.423514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc4166-61f2-4f9e-af45-8e98b7ae749e None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:05.424180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc4166-61f2-4f9e-af45-8e98b7ae749e None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:05.424486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc4166-61f2-4f9e-af45-8e98b7ae749e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:05.425049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ad85c-b59a-40b8-8f72-084acbd3c111 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:05.426323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc4166-61f2-4f9e-af45-8e98b7ae749e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-ba3ad85c-b59a-40b8-8f72-084acbd3c111 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:05.435408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:05.436495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 03:31:05.437417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:05.440240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 483/957] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:31:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:05.448295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ad85c-b59a-40b8-8f72-084acbd3c111 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:05.450775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 475/958] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:31:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:05.486646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:05.585439 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ad85c-b59a-40b8-8f72-084acbd3c111 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:05.627669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1aac8-3401-42de-abf9-0bcf1e8bc3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:05.628208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1aac8-3401-42de-abf9-0bcf1e8bc3de None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:05.628487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1aac8-3401-42de-abf9-0bcf1e8bc3de None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:05.628760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1aac8-3401-42de-abf9-0bcf1e8bc3de None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:05.629404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1aac8-3401-42de-abf9-0bcf1e8bc3de None admin] RBAC: Adding query filter params (name=compute) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:05.629705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1aac8-3401-42de-abf9-0bcf1e8bc3de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:05.631156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1aac8-3401-42de-abf9-0bcf1e8bc3de None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:05.638222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 484/959] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:31:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:05.644760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92682d99-a1c0-4222-a69f-6335484efdce None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:05.645051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92682d99-a1c0-4222-a69f-6335484efdce None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:05.645321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92682d99-a1c0-4222-a69f-6335484efdce None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:05.647504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 476/960] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:05.647951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1aac8-3401-42de-abf9-0bcf1e8bc3de None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:05.655034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce77ed5-86a0-4733-a9f3-39f82c351b79 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:05.655439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce77ed5-86a0-4733-a9f3-39f82c351b79 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:05.655646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce77ed5-86a0-4733-a9f3-39f82c351b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:05.688482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-fce77ed5-86a0-4733-a9f3-39f82c351b79 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:05.691373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fce77ed5-86a0-4733-a9f3-39f82c351b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:05.795072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:05.809267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 477/961] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:05.813771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1aac8-3401-42de-abf9-0bcf1e8bc3de None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:05.868956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296ac8-5e18-42d6-a6ea-f8fd933e7700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:05.868956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296ac8-5e18-42d6-a6ea-f8fd933e7700 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:05.868956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296ac8-5e18-42d6-a6ea-f8fd933e7700 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:05.868956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296ac8-5e18-42d6-a6ea-f8fd933e7700 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:05.869994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296ac8-5e18-42d6-a6ea-f8fd933e7700 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:05.869994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296ac8-5e18-42d6-a6ea-f8fd933e7700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:05.869994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296ac8-5e18-42d6-a6ea-f8fd933e7700 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:05.875335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 485/962] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:31:05 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:06.124268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce77ed5-86a0-4733-a9f3-39f82c351b79 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:06.124268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:06.237079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce77ed5-86a0-4733-a9f3-39f82c351b79 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:06.273144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db3796-c83f-4692-96f7-eca51c996cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:06.273998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db3796-c83f-4692-96f7-eca51c996cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:06.274399 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db3796-c83f-4692-96f7-eca51c996cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:06.274895 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db3796-c83f-4692-96f7-eca51c996cf0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:06.286173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db3796-c83f-4692-96f7-eca51c996cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-59db3796-c83f-4692-96f7-eca51c996cf0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:06.293767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:06.294721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:31:06.295593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:06.298054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 478/963] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Sep 17 03:31:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:06.299485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296ac8-5e18-42d6-a6ea-f8fd933e7700 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:06.416761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296ac8-5e18-42d6-a6ea-f8fd933e7700 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:06.455292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50fc34-8891-4b95-ad93-7eaf434f6027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:06.457185 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50fc34-8891-4b95-ad93-7eaf434f6027 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:06.457457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50fc34-8891-4b95-ad93-7eaf434f6027 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:06.457804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50fc34-8891-4b95-ad93-7eaf434f6027 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:06.458480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50fc34-8891-4b95-ad93-7eaf434f6027 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:06.458781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50fc34-8891-4b95-ad93-7eaf434f6027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:06.460149 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50fc34-8891-4b95-ad93-7eaf434f6027 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:06.470138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 486/964] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Sep 17 03:31:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:06.476001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db3796-c83f-4692-96f7-eca51c996cf0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:06.550169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:06.598935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db3796-c83f-4692-96f7-eca51c996cf0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:06.644636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5a977-532a-41ef-8a2b-3c1f4d9f4054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:06.645274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5a977-532a-41ef-8a2b-3c1f4d9f4054 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:06.645553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5a977-532a-41ef-8a2b-3c1f4d9f4054 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:06.645854 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5a977-532a-41ef-8a2b-3c1f4d9f4054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:06.659497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5a977-532a-41ef-8a2b-3c1f4d9f4054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:06.660958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5a977-532a-41ef-8a2b-3c1f4d9f4054 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d6c5a977-532a-41ef-8a2b-3c1f4d9f4054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:06.676153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:06.677134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:06.678077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:06.678077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:06.678077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:06.684261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 479/965] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:06.686450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50fc34-8891-4b95-ad93-7eaf434f6027 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:06.711844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:06.747742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856ce1-6719-4310-a56d-4c89701b6e4a None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:06.749194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856ce1-6719-4310-a56d-4c89701b6e4a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:06.749455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856ce1-6719-4310-a56d-4c89701b6e4a None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:06.752238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 480/966] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:06.757572 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30037c-45e0-4933-95c2-b2542efa5816 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:06.757857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30037c-45e0-4933-95c2-b2542efa5816 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:06.758214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30037c-45e0-4933-95c2-b2542efa5816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:06.791299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1b30037c-45e0-4933-95c2-b2542efa5816 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:06.792513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1b30037c-45e0-4933-95c2-b2542efa5816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:06.836936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50fc34-8891-4b95-ad93-7eaf434f6027 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:06.917817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5c3ae-a0e4-46cb-87de-a799f03ecf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:06.917817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5c3ae-a0e4-46cb-87de-a799f03ecf54 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:06.917817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5c3ae-a0e4-46cb-87de-a799f03ecf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:06.917817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5c3ae-a0e4-46cb-87de-a799f03ecf54 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:06.918679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5c3ae-a0e4-46cb-87de-a799f03ecf54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:06.918679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5c3ae-a0e4-46cb-87de-a799f03ecf54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:06.918842 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 481/967] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:06.919201 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5c3ae-a0e4-46cb-87de-a799f03ecf54 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:06.936361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 487/968] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:06.942196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30037c-45e0-4933-95c2-b2542efa5816 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:07.006110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:07.051421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30037c-45e0-4933-95c2-b2542efa5816 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:07.088357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb00d7-9e6e-424f-8d01-c2a22a9c78ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:07.089126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb00d7-9e6e-424f-8d01-c2a22a9c78ad None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:07.089498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb00d7-9e6e-424f-8d01-c2a22a9c78ad None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:07.089899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb00d7-9e6e-424f-8d01-c2a22a9c78ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:07.109574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb00d7-9e6e-424f-8d01-c2a22a9c78ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:07.111038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb00d7-9e6e-424f-8d01-c2a22a9c78ad None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-2dfb00d7-9e6e-424f-8d01-c2a22a9c78ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:31:07.131199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:07.132183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:07.133719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 482/969] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Sep 17 03:31:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:07.147193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5c3ae-a0e4-46cb-87de-a799f03ecf54 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:07.261822 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb00d7-9e6e-424f-8d01-c2a22a9c78ad None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:07.282130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:07.316794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5c3ae-a0e4-46cb-87de-a799f03ecf54 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:07.379977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148298c-6daa-43ce-be98-694402db61c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:07.385762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148298c-6daa-43ce-be98-694402db61c1 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:07.385762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148298c-6daa-43ce-be98-694402db61c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:07.385762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148298c-6daa-43ce-be98-694402db61c1 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:07.385762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148298c-6daa-43ce-be98-694402db61c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:07.385762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148298c-6daa-43ce-be98-694402db61c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:07.385762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148298c-6daa-43ce-be98-694402db61c1 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:07.425754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 488/970] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Sep 17 03:31:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:07.433760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148298c-6daa-43ce-be98-694402db61c1 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:07.469759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb00d7-9e6e-424f-8d01-c2a22a9c78ad None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:07.473749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:07.529131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37971b0f-0a7f-408c-a1c9-4abf5a4cf80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:07.533746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37971b0f-0a7f-408c-a1c9-4abf5a4cf80c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:07.533746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37971b0f-0a7f-408c-a1c9-4abf5a4cf80c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:07.533746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37971b0f-0a7f-408c-a1c9-4abf5a4cf80c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:07.533746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37971b0f-0a7f-408c-a1c9-4abf5a4cf80c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:07.537747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37971b0f-0a7f-408c-a1c9-4abf5a4cf80c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-37971b0f-0a7f-408c-a1c9-4abf5a4cf80c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:07.553751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:07.555672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:07.557137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 483/971] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:31:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:07.565695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37971b0f-0a7f-408c-a1c9-4abf5a4cf80c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:07.636254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148298c-6daa-43ce-be98-694402db61c1 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:07.725758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac2d98-0156-4bc0-bba2-32c8f1fcfe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:07.725758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac2d98-0156-4bc0-bba2-32c8f1fcfe72 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:07.725758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac2d98-0156-4bc0-bba2-32c8f1fcfe72 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:07.725758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac2d98-0156-4bc0-bba2-32c8f1fcfe72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:07.733643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:07.770412 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37971b0f-0a7f-408c-a1c9-4abf5a4cf80c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:07.819379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bf0f8-6983-4ed6-97f5-996699488d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:07.820085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bf0f8-6983-4ed6-97f5-996699488d26 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:07.820438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bf0f8-6983-4ed6-97f5-996699488d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:07.821317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bf0f8-6983-4ed6-97f5-996699488d26 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:07.822096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bf0f8-6983-4ed6-97f5-996699488d26 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:07.822458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bf0f8-6983-4ed6-97f5-996699488d26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:07.824582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bf0f8-6983-4ed6-97f5-996699488d26 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:07.831563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 484/972] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Sep 17 03:31:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:07.837812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bf0f8-6983-4ed6-97f5-996699488d26 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:07.880734 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac2d98-0156-4bc0-bba2-32c8f1fcfe72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:07.882974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac2d98-0156-4bc0-bba2-32c8f1fcfe72 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:07.886808 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:07.986520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bf0f8-6983-4ed6-97f5-996699488d26 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:08.038092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744b389-6cae-4db7-a57b-892f544b370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:08.038726 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744b389-6cae-4db7-a57b-892f544b370c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.039007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744b389-6cae-4db7-a57b-892f544b370c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.039279 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744b389-6cae-4db7-a57b-892f544b370c None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.040684 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744b389-6cae-4db7-a57b-892f544b370c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:08.041559 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744b389-6cae-4db7-a57b-892f544b370c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:08.042973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744b389-6cae-4db7-a57b-892f544b370c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:08.050326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 485/973] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Sep 17 03:31:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:08.052252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb5a1f-9e2b-47b1-b69f-258c41c70cac None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.053189 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb5a1f-9e2b-47b1-b69f-258c41c70cac None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.053976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb5a1f-9e2b-47b1-b69f-258c41c70cac None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.055588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 486/974] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:08.056603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb5a1f-9e2b-47b1-b69f-258c41c70cac None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:08.056971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 489/975] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:31:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 03:31:08.063225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61066edd-9166-46d2-84a0-d11217e1a82c None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.064162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61066edd-9166-46d2-84a0-d11217e1a82c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.064763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61066edd-9166-46d2-84a0-d11217e1a82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.085104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-61066edd-9166-46d2-84a0-d11217e1a82c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:08.086911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-61066edd-9166-46d2-84a0-d11217e1a82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:08.118173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:08.167203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 490/976] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:08.168384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61066edd-9166-46d2-84a0-d11217e1a82c None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:08.183190 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:08.194092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb5a1f-9e2b-47b1-b69f-258c41c70cac None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:08.250336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9c71c-a2ed-46dc-988e-9574a826fc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:08.250961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9c71c-a2ed-46dc-988e-9574a826fc8f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.251243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9c71c-a2ed-46dc-988e-9574a826fc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.251517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9c71c-a2ed-46dc-988e-9574a826fc8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.252239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9c71c-a2ed-46dc-988e-9574a826fc8f None admin] RBAC: Adding query filter params (interface=admin, service_id=973838b71feb4dc8a27d4d1654567523, region_id=RegionOne) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:08.252550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9c71c-a2ed-46dc-988e-9574a826fc8f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:08.254317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9c71c-a2ed-46dc-988e-9574a826fc8f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:08.261345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 487/977] 10.222.0.42 () {56 vars in 1249 bytes} [Sat Sep 17 03:31:08 2022] GET /identity/v3/endpoints?service_id=973838b71feb4dc8a27d4d1654567523&interface=admin®ion_id=RegionOne => generated 193 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:08.271000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9c71c-a2ed-46dc-988e-9574a826fc8f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:08.312557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61066edd-9166-46d2-84a0-d11217e1a82c None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:08.375195 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86038430-0ba7-4449-8181-9e93d0984559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:08.376769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86038430-0ba7-4449-8181-9e93d0984559 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.377053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86038430-0ba7-4449-8181-9e93d0984559 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.377327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86038430-0ba7-4449-8181-9e93d0984559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.392132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86038430-0ba7-4449-8181-9e93d0984559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:08.393587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86038430-0ba7-4449-8181-9e93d0984559 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:08.413065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9c71c-a2ed-46dc-988e-9574a826fc8f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-86038430-0ba7-4449-8181-9e93d0984559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:08.413247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:08.414216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:08.415707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:08.415707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:08.415707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:08.415707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 491/978] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:08.417222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dac385-6fb0-4154-80e5-11347dc737a1 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.417222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dac385-6fb0-4154-80e5-11347dc737a1 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.417504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dac385-6fb0-4154-80e5-11347dc737a1 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.424730 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 492/979] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:08.424730 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dac385-6fb0-4154-80e5-11347dc737a1 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:08.441237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:08.468365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc643a61-e685-4395-aeda-7b190ce8b41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:08.468972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc643a61-e685-4395-aeda-7b190ce8b41f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.469254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc643a61-e685-4395-aeda-7b190ce8b41f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.469530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc643a61-e685-4395-aeda-7b190ce8b41f None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.470171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc643a61-e685-4395-aeda-7b190ce8b41f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:08.471549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc643a61-e685-4395-aeda-7b190ce8b41f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:08.478053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 488/980] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Sep 17 03:31:08 2022] GET /identity/v3/services => generated 1497 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:31:08.479605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d005-a814-42af-b7c6-d6bd89bbb348 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.479943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d005-a814-42af-b7c6-d6bd89bbb348 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.480257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d005-a814-42af-b7c6-d6bd89bbb348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.502584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1a86d005-a814-42af-b7c6-d6bd89bbb348 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:08.503895 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1a86d005-a814-42af-b7c6-d6bd89bbb348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:08.529522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:08.562143 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dac385-6fb0-4154-80e5-11347dc737a1 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:08.602635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 489/981] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:08.602635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86d005-a814-42af-b7c6-d6bd89bbb348 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:08.605689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed3f97-47f7-492d-bd6c-0f7b7dbb6454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:08.606429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed3f97-47f7-492d-bd6c-0f7b7dbb6454 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.606785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed3f97-47f7-492d-bd6c-0f7b7dbb6454 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.607207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed3f97-47f7-492d-bd6c-0f7b7dbb6454 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.607921 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed3f97-47f7-492d-bd6c-0f7b7dbb6454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:08.608308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed3f97-47f7-492d-bd6c-0f7b7dbb6454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:08.611916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed3f97-47f7-492d-bd6c-0f7b7dbb6454 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:08.625692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 493/982] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:08.631948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed3f97-47f7-492d-bd6c-0f7b7dbb6454 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:08.752237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:08.759503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86d005-a814-42af-b7c6-d6bd89bbb348 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:08.790923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed3f97-47f7-492d-bd6c-0f7b7dbb6454 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:08.808429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233493cc-ead1-4cbe-a2bd-6fb8bf0b0c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:08.809371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233493cc-ead1-4cbe-a2bd-6fb8bf0b0c40 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.809886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233493cc-ead1-4cbe-a2bd-6fb8bf0b0c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.810357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233493cc-ead1-4cbe-a2bd-6fb8bf0b0c40 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.811212 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233493cc-ead1-4cbe-a2bd-6fb8bf0b0c40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:08.812837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233493cc-ead1-4cbe-a2bd-6fb8bf0b0c40 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:08.853859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82494b-41d2-46fa-a8ec-5be94a7dd18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:08.853859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82494b-41d2-46fa-a8ec-5be94a7dd18e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:08.854346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82494b-41d2-46fa-a8ec-5be94a7dd18e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:08.854915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82494b-41d2-46fa-a8ec-5be94a7dd18e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:08.862967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82494b-41d2-46fa-a8ec-5be94a7dd18e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:08.864761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82494b-41d2-46fa-a8ec-5be94a7dd18e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-9b82494b-41d2-46fa-a8ec-5be94a7dd18e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:31:08.877583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:08.878729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:08.879655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:31:08.879655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:31:08.879655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:31:08.879655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:31:08.879655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:31:08.879655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:31:08.879655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:08.879655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:08.882113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 494/983] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:31:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:08.889057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.sql.core [None req-233493cc-ead1-4cbe-a2bd-6fb8bf0b0c40 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 03:31:08.889057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 17 03:31:08.889057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [parameters: {'user_id': '701bd49cfed247bda3365464e14b5454', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 17 03:31:08.889057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81725) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 03:31:08.889423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82494b-41d2-46fa-a8ec-5be94a7dd18e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-233493cc-ead1-4cbe-a2bd-6fb8bf0b0c40 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application conn.query(q) Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result.read() Sep 17 03:31:08.897247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 03:31:08.898236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application next(self.gen) Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application yield resource Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application next(self.gen) Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise self.value Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application session.commit() Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 03:31:08.899754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.session.flush() Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application compat.raise_( Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 03:31:08.902083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 03:31:08.904287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application conn.query(q) Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result.read() Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 03:31:08.906281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [parameters: {'user_id': '701bd49cfed247bda3365464e14b5454', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:08.910346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 17 03:31:08.913077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 17 03:31:08.915736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 17 03:31:08.915736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 17 03:31:08.915736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 03:31:08.915736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 03:31:08.915736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 17 03:31:08.915736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:08.915736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 490/984] 10.222.0.42 () {60 vars in 1131 bytes} [Sat Sep 17 03:31:08 2022] POST /identity/v3/users => generated 160 bytes in 306 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:08.915736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233493cc-ead1-4cbe-a2bd-6fb8bf0b0c40 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:08.938120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:09.041753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82494b-41d2-46fa-a8ec-5be94a7dd18e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:09.045768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233493cc-ead1-4cbe-a2bd-6fb8bf0b0c40 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:09.075581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8d051-d81b-4bd8-8188-f010789076e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:09.076868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8d051-d81b-4bd8-8188-f010789076e7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:09.077344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8d051-d81b-4bd8-8188-f010789076e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:09.077975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8d051-d81b-4bd8-8188-f010789076e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:09.078856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8d051-d81b-4bd8-8188-f010789076e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:09.079162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8d051-d81b-4bd8-8188-f010789076e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:09.080598 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8d051-d81b-4bd8-8188-f010789076e7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:09.084618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:09.102014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 495/985] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:09.106302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8d051-d81b-4bd8-8188-f010789076e7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:09.108922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cc7e0-604a-47b1-8754-76705cd69384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:09.109620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cc7e0-604a-47b1-8754-76705cd69384 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:09.109933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cc7e0-604a-47b1-8754-76705cd69384 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:09.110211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cc7e0-604a-47b1-8754-76705cd69384 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:09.121068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cc7e0-604a-47b1-8754-76705cd69384 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:09.123207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cc7e0-604a-47b1-8754-76705cd69384 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-269cc7e0-604a-47b1-8754-76705cd69384 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:31:09.140806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:09.141862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:09.142934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:09.142934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:09.142934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:31:09.142934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:31:09.142934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:31:09.142934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:31:09.142934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:31:09.142934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:31:09.142934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 03:31:09.142934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:09.150799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 491/986] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:31:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:09.150799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cc7e0-604a-47b1-8754-76705cd69384 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:09.246247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:09.248659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8d051-d81b-4bd8-8188-f010789076e7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:09.277639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cc7e0-604a-47b1-8754-76705cd69384 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:09.296002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732434bd-4c6c-4ca9-902a-1d7e2d71042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:09.296816 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732434bd-4c6c-4ca9-902a-1d7e2d71042c None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:09.297296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732434bd-4c6c-4ca9-902a-1d7e2d71042c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:09.297658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732434bd-4c6c-4ca9-902a-1d7e2d71042c None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:09.298676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732434bd-4c6c-4ca9-902a-1d7e2d71042c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:09.300225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732434bd-4c6c-4ca9-902a-1d7e2d71042c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:09.319680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0739f-a74e-4970-84ae-c23f894edd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:09.324836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0739f-a74e-4970-84ae-c23f894edd60 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:09.324836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0739f-a74e-4970-84ae-c23f894edd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:09.324836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0739f-a74e-4970-84ae-c23f894edd60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:09.324836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0739f-a74e-4970-84ae-c23f894edd60 None admin] RBAC: Adding query filter params (role.id=e88868b1e3fc44e6aee8a63eb09ddd9b, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=b3803e9daa1848ac9feec1bdef4130de) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:09.324836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0739f-a74e-4970-84ae-c23f894edd60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:09.324836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0739f-a74e-4970-84ae-c23f894edd60 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:09.334071 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 492/987] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:31:09 2022] GET /identity/v3/role_assignments?user.id=b3803e9daa1848ac9feec1bdef4130de&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e88868b1e3fc44e6aee8a63eb09ddd9b => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:09.335243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0739f-a74e-4970-84ae-c23f894edd60 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:09.394592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 496/988] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:31:09 2022] POST /identity/v3/services => generated 242 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:09.491049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0739f-a74e-4970-84ae-c23f894edd60 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:09.549431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16b63d-4cc2-4dee-b6ac-1036185744f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:09.550088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16b63d-4cc2-4dee-b6ac-1036185744f5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:09.550365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16b63d-4cc2-4dee-b6ac-1036185744f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:09.550638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16b63d-4cc2-4dee-b6ac-1036185744f5 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:09.551351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16b63d-4cc2-4dee-b6ac-1036185744f5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:09.551652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16b63d-4cc2-4dee-b6ac-1036185744f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:09.553103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16b63d-4cc2-4dee-b6ac-1036185744f5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:09.565241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:09.595833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 493/989] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Sep 17 03:31:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:10.813295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7e2b9-dfd5-4cbd-855f-9b91818f8e90 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:10.813927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7e2b9-dfd5-4cbd-855f-9b91818f8e90 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:10.813927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7e2b9-dfd5-4cbd-855f-9b91818f8e90 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:10.816397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 497/990] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:10.821756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fb027-dc22-4ee5-8fa9-681ba37f57cd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:10.822951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fb027-dc22-4ee5-8fa9-681ba37f57cd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:10.822951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fb027-dc22-4ee5-8fa9-681ba37f57cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:10.843809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-300fb027-dc22-4ee5-8fa9-681ba37f57cd None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:10.845849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-300fb027-dc22-4ee5-8fa9-681ba37f57cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:10.925855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 494/991] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:11.198528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7e2b9-dfd5-4cbd-855f-9b91818f8e90 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:11.266266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:11.320518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7e2b9-dfd5-4cbd-855f-9b91818f8e90 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:11.363531 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b47d4b-8c17-4431-b53d-92d7ca8c2c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:11.364165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b47d4b-8c17-4431-b53d-92d7ca8c2c24 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:11.364444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b47d4b-8c17-4431-b53d-92d7ca8c2c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:11.364718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b47d4b-8c17-4431-b53d-92d7ca8c2c24 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:11.365369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b47d4b-8c17-4431-b53d-92d7ca8c2c24 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:11.366790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b47d4b-8c17-4431-b53d-92d7ca8c2c24 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-42b47d4b-8c17-4431-b53d-92d7ca8c2c24 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:11.377236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:11.378287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:11.379151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 498/992] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Sep 17 03:31:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:11.386005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fb027-dc22-4ee5-8fa9-681ba37f57cd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:11.503383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fb027-dc22-4ee5-8fa9-681ba37f57cd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:11.530609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:11.543413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f94da4-ab27-4cd7-a615-67f18134967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:11.544328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f94da4-ab27-4cd7-a615-67f18134967b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:11.544765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f94da4-ab27-4cd7-a615-67f18134967b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:11.545223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f94da4-ab27-4cd7-a615-67f18134967b None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:11.546045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f94da4-ab27-4cd7-a615-67f18134967b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:11.546500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f94da4-ab27-4cd7-a615-67f18134967b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:11.548110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f94da4-ab27-4cd7-a615-67f18134967b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:11.560337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b47d4b-8c17-4431-b53d-92d7ca8c2c24 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:11.561616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 495/993] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Sep 17 03:31:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:11.689467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b47d4b-8c17-4431-b53d-92d7ca8c2c24 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:11.715233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:11.728743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de08ed-213d-4594-bfec-7d8b885913ea None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:11.737534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de08ed-213d-4594-bfec-7d8b885913ea None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:11.737534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de08ed-213d-4594-bfec-7d8b885913ea None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:11.737534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 496/994] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:11.737821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae7da6-ea4e-488f-85dd-6d48dba97fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:11.737821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae7da6-ea4e-488f-85dd-6d48dba97fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:11.737821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae7da6-ea4e-488f-85dd-6d48dba97fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:11.737821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae7da6-ea4e-488f-85dd-6d48dba97fc1 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:11.738112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7e7f7-a202-483f-8319-fd07644ea785 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:11.738187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae7da6-ea4e-488f-85dd-6d48dba97fc1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:11.738257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7e7f7-a202-483f-8319-fd07644ea785 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:11.738326 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae7da6-ea4e-488f-85dd-6d48dba97fc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:11.738420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7e7f7-a202-483f-8319-fd07644ea785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:11.738750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae7da6-ea4e-488f-85dd-6d48dba97fc1 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:11.750661 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 499/995] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Sep 17 03:31:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:11.751749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae7da6-ea4e-488f-85dd-6d48dba97fc1 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:11.761094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b1b7e7f7-a202-483f-8319-fd07644ea785 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:11.762255 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b1b7e7f7-a202-483f-8319-fd07644ea785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:11.857653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 497/996] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:11.858823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34e3f4-ce32-4123-9a65-1fcdb9671ab0 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:11.859116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34e3f4-ce32-4123-9a65-1fcdb9671ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:11.859394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34e3f4-ce32-4123-9a65-1fcdb9671ab0 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:11.860778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 498/997] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:11.866019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f9f10-d0bd-4998-ae1b-837b210636e5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:11.866305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f9f10-d0bd-4998-ae1b-837b210636e5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:11.866580 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f9f10-d0bd-4998-ae1b-837b210636e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:11.869399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae7da6-ea4e-488f-85dd-6d48dba97fc1 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:11.891130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-421f9f10-d0bd-4998-ae1b-837b210636e5 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:11.896814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-421f9f10-d0bd-4998-ae1b-837b210636e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:11.915695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62b47c-8992-4051-a2e9-4073eed63035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:11.916532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b47c-8992-4051-a2e9-4073eed63035 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:11.916811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b47c-8992-4051-a2e9-4073eed63035 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:11.917076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b47c-8992-4051-a2e9-4073eed63035 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:11.918231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62b47c-8992-4051-a2e9-4073eed63035 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:11.919890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62b47c-8992-4051-a2e9-4073eed63035 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:11.975712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 499/998] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:11.991239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 500/999] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:31:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:12.134253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f9f10-d0bd-4998-ae1b-837b210636e5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:12.189751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:12.250965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f9f10-d0bd-4998-ae1b-837b210636e5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:12.257150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62b47c-8992-4051-a2e9-4073eed63035 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:12.302729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75316dbb-75c3-43d6-9ef3-202ceb2d0525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:12.304260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75316dbb-75c3-43d6-9ef3-202ceb2d0525 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:12.304760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75316dbb-75c3-43d6-9ef3-202ceb2d0525 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:12.305542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75316dbb-75c3-43d6-9ef3-202ceb2d0525 None admin] PATH_INFO: `/v3/services/image` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:12.306747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75316dbb-75c3-43d6-9ef3-202ceb2d0525 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:12.309786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75316dbb-75c3-43d6-9ef3-202ceb2d0525 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:12.310213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-75316dbb-75c3-43d6-9ef3-202ceb2d0525 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:12.319965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:12.324980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:12.325972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 500/1000] 10.222.0.42 () {56 vars in 1105 bytes} [Sat Sep 17 03:31:12 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:12.328107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75316dbb-75c3-43d6-9ef3-202ceb2d0525 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:12.389832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62b47c-8992-4051-a2e9-4073eed63035 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:12.434983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d481564-dfb4-4562-b04b-ec9610374a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:12.436045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d481564-dfb4-4562-b04b-ec9610374a5a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:12.436890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d481564-dfb4-4562-b04b-ec9610374a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:12.437162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d481564-dfb4-4562-b04b-ec9610374a5a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:12.438043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d481564-dfb4-4562-b04b-ec9610374a5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:12.439602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d481564-dfb4-4562-b04b-ec9610374a5a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-4d481564-dfb4-4562-b04b-ec9610374a5a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:12.451771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:12.452839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:12.454238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 501/1001] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:31:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:12.462334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d481564-dfb4-4562-b04b-ec9610374a5a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:12.463160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75316dbb-75c3-43d6-9ef3-202ceb2d0525 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:12.473748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:12.506196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb749a0-f933-412c-b715-56565f94fa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:12.507170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb749a0-f933-412c-b715-56565f94fa50 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:12.507716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb749a0-f933-412c-b715-56565f94fa50 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:12.508965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb749a0-f933-412c-b715-56565f94fa50 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:12.509975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb749a0-f933-412c-b715-56565f94fa50 None admin] RBAC: Adding query filter params (name=image) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:12.510513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb749a0-f933-412c-b715-56565f94fa50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:12.512379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb749a0-f933-412c-b715-56565f94fa50 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:12.518507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 501/1002] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:12.523786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb749a0-f933-412c-b715-56565f94fa50 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:12.597753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d481564-dfb4-4562-b04b-ec9610374a5a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:12.606637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:12.640529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb749a0-f933-412c-b715-56565f94fa50 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:12.644470 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21c42a-de44-4ff1-bd0d-f53b3f6185e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:12.646339 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21c42a-de44-4ff1-bd0d-f53b3f6185e4 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:12.647000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21c42a-de44-4ff1-bd0d-f53b3f6185e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:12.647474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21c42a-de44-4ff1-bd0d-f53b3f6185e4 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:12.649392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21c42a-de44-4ff1-bd0d-f53b3f6185e4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:12.650218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21c42a-de44-4ff1-bd0d-f53b3f6185e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:12.652429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21c42a-de44-4ff1-bd0d-f53b3f6185e4 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:12.659500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 502/1003] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:31:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:12.665013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21c42a-de44-4ff1-bd0d-f53b3f6185e4 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:12.684026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620153c-66e5-47b5-9cc2-805da78f07ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:12.685085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620153c-66e5-47b5-9cc2-805da78f07ee None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:12.685366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620153c-66e5-47b5-9cc2-805da78f07ee None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:12.685638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620153c-66e5-47b5-9cc2-805da78f07ee None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:12.686625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620153c-66e5-47b5-9cc2-805da78f07ee None admin] RBAC: Adding query filter params (type=image) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:12.686927 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620153c-66e5-47b5-9cc2-805da78f07ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:12.688722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620153c-66e5-47b5-9cc2-805da78f07ee None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:12.696740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 502/1004] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:12.812128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21c42a-de44-4ff1-bd0d-f53b3f6185e4 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:12.858208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ecaca-33bb-46a3-a073-66a5dab9108b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:12.859275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ecaca-33bb-46a3-a073-66a5dab9108b None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:12.859800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ecaca-33bb-46a3-a073-66a5dab9108b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:12.860221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ecaca-33bb-46a3-a073-66a5dab9108b None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:12.861074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ecaca-33bb-46a3-a073-66a5dab9108b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:12.861528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ecaca-33bb-46a3-a073-66a5dab9108b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:12.863339 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ecaca-33bb-46a3-a073-66a5dab9108b None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:12.870708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 503/1005] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:31:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:12.876374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620153c-66e5-47b5-9cc2-805da78f07ee None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:12.925042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:12.995763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620153c-66e5-47b5-9cc2-805da78f07ee None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:13.037438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bfe66-3be1-4f2a-97bc-a57f2af4d1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:13.038344 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bfe66-3be1-4f2a-97bc-a57f2af4d1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:13.038778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bfe66-3be1-4f2a-97bc-a57f2af4d1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:13.039229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bfe66-3be1-4f2a-97bc-a57f2af4d1a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:13.040210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bfe66-3be1-4f2a-97bc-a57f2af4d1a4 None admin] RBAC: Adding query filter params (interface=public, service_id=15f1271fea6246f3845bbf3be99dd4c5, region_id=RegionOne) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:13.040674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bfe66-3be1-4f2a-97bc-a57f2af4d1a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:13.042306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bfe66-3be1-4f2a-97bc-a57f2af4d1a4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:13.048775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 503/1006] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Sep 17 03:31:12 2022] GET /identity/v3/endpoints?service_id=15f1271fea6246f3845bbf3be99dd4c5&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:13.058030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ecaca-33bb-46a3-a073-66a5dab9108b None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:13.109508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:13.184871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ecaca-33bb-46a3-a073-66a5dab9108b None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:13.223414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc74d5-03b6-4f5b-8930-0ef1f68ae8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:13.224020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc74d5-03b6-4f5b-8930-0ef1f68ae8fc None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:13.224292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc74d5-03b6-4f5b-8930-0ef1f68ae8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:13.224555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc74d5-03b6-4f5b-8930-0ef1f68ae8fc None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:13.225151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc74d5-03b6-4f5b-8930-0ef1f68ae8fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:13.226596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc74d5-03b6-4f5b-8930-0ef1f68ae8fc None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:13.234503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 504/1007] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Sep 17 03:31:13 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:31:14.290532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9ebcb-885b-4d38-9390-31af8cabee71 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:14.291077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9ebcb-885b-4d38-9390-31af8cabee71 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:14.291077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9ebcb-885b-4d38-9390-31af8cabee71 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:14.292773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 504/1008] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:14.298469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68fab4-8be0-4bd5-9aa9-6df779463b28 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:14.298885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68fab4-8be0-4bd5-9aa9-6df779463b28 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:14.299027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68fab4-8be0-4bd5-9aa9-6df779463b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:14.318693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-9d68fab4-8be0-4bd5-9aa9-6df779463b28 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:14.319890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9d68fab4-8be0-4bd5-9aa9-6df779463b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:14.400468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 505/1009] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:14.675838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9ebcb-885b-4d38-9390-31af8cabee71 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:14.781365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:14.788579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9ebcb-885b-4d38-9390-31af8cabee71 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:14.836968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f933fe-468b-43fb-9f61-0bdb53e5c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:14.837712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f933fe-468b-43fb-9f61-0bdb53e5c816 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:14.838015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f933fe-468b-43fb-9f61-0bdb53e5c816 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:14.838286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f933fe-468b-43fb-9f61-0bdb53e5c816 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:14.839707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f933fe-468b-43fb-9f61-0bdb53e5c816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:14.841456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f933fe-468b-43fb-9f61-0bdb53e5c816 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:14.845816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 505/1010] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:14.854161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68fab4-8be0-4bd5-9aa9-6df779463b28 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:14.932784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:14.977964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a136c-6fd7-44ab-8dfc-eb13d8cfab48 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:14.978387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a136c-6fd7-44ab-8dfc-eb13d8cfab48 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:14.978786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a136c-6fd7-44ab-8dfc-eb13d8cfab48 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:14.980653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 506/1011] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:31:14.985448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5faed-0743-4ba0-b600-6cbbaeb408d1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:14.985921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5faed-0743-4ba0-b600-6cbbaeb408d1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:14.986034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5faed-0743-4ba0-b600-6cbbaeb408d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:15.016211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b1c5faed-0743-4ba0-b600-6cbbaeb408d1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:15.017577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b1c5faed-0743-4ba0-b600-6cbbaeb408d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:15.029108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68fab4-8be0-4bd5-9aa9-6df779463b28 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:15.069938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a3d89-f351-407e-a6be-0618b8a2f328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:15.071305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a3d89-f351-407e-a6be-0618b8a2f328 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:15.072167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a3d89-f351-407e-a6be-0618b8a2f328 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:15.072451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a3d89-f351-407e-a6be-0618b8a2f328 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:15.073326 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a3d89-f351-407e-a6be-0618b8a2f328 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:15.075628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a3d89-f351-407e-a6be-0618b8a2f328 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:15.127148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 507/1012] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:15.165955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 506/1013] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:31:14 2022] POST /identity/v3/projects => generated 311 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:15.410196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5faed-0743-4ba0-b600-6cbbaeb408d1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:15.485307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3b999-aa13-4b58-a8e0-ace2e1478d2e None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:15.485836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3b999-aa13-4b58-a8e0-ace2e1478d2e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:15.485915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3b999-aa13-4b58-a8e0-ace2e1478d2e None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:15.487243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 507/1014] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:15.497795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f50f-e6d2-4973-9ed8-68d8809bc2dd None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:15.497795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f50f-e6d2-4973-9ed8-68d8809bc2dd None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:15.497795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f50f-e6d2-4973-9ed8-68d8809bc2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:15.514637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-acd4f50f-e6d2-4973-9ed8-68d8809bc2dd None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:15.515838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-acd4f50f-e6d2-4973-9ed8-68d8809bc2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:15.529684 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:15.542748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5faed-0743-4ba0-b600-6cbbaeb408d1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:15.583672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd088d22-be19-4c99-8694-b3be648be9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:15.584283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd088d22-be19-4c99-8694-b3be648be9b8 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:15.584605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd088d22-be19-4c99-8694-b3be648be9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:15.584880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd088d22-be19-4c99-8694-b3be648be9b8 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:15.585525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd088d22-be19-4c99-8694-b3be648be9b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:15.587667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd088d22-be19-4c99-8694-b3be648be9b8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:15.600808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 508/1015] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:15.701845 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 508/1016] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:31:15 2022] POST /identity/v3/services => generated 242 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:15.898353 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4f50f-e6d2-4973-9ed8-68d8809bc2dd None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:15.914506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:16.048663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4f50f-e6d2-4973-9ed8-68d8809bc2dd None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:16.114049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb25b8b-f655-4b39-8021-242c8de922c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:16.114713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb25b8b-f655-4b39-8021-242c8de922c4 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:16.114995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb25b8b-f655-4b39-8021-242c8de922c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:16.115273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb25b8b-f655-4b39-8021-242c8de922c4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:16.115864 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb25b8b-f655-4b39-8021-242c8de922c4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:16.117833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb25b8b-f655-4b39-8021-242c8de922c4 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-3fb25b8b-f655-4b39-8021-242c8de922c4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:16.127623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:16.128656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 03:31:16.129523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:16.132781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 509/1017] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:31:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:16.140144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd088d22-be19-4c99-8694-b3be648be9b8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:16.260083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd088d22-be19-4c99-8694-b3be648be9b8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:16.300094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f463fbd-1690-4f4b-8065-fbdbeedd6f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:16.300617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f463fbd-1690-4f4b-8065-fbdbeedd6f4c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:16.300904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f463fbd-1690-4f4b-8065-fbdbeedd6f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:16.301186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f463fbd-1690-4f4b-8065-fbdbeedd6f4c None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:16.301855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f463fbd-1690-4f4b-8065-fbdbeedd6f4c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:16.302164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f463fbd-1690-4f4b-8065-fbdbeedd6f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:16.303532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f463fbd-1690-4f4b-8065-fbdbeedd6f4c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:16.309414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 509/1018] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:31:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:16.314859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb25b8b-f655-4b39-8021-242c8de922c4 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:16.451170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:16.482068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb25b8b-f655-4b39-8021-242c8de922c4 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:16.519540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8fe38-a074-4fb1-9f18-4b34ee9756ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:16.520915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8fe38-a074-4fb1-9f18-4b34ee9756ac None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:16.521380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8fe38-a074-4fb1-9f18-4b34ee9756ac None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:16.521870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8fe38-a074-4fb1-9f18-4b34ee9756ac None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:16.522679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8fe38-a074-4fb1-9f18-4b34ee9756ac None admin] RBAC: Adding query filter params (type=compute) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:16.523199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8fe38-a074-4fb1-9f18-4b34ee9756ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:16.524933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8fe38-a074-4fb1-9f18-4b34ee9756ac None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:16.530306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 510/1019] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Sep 17 03:31:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:16.536130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f463fbd-1690-4f4b-8065-fbdbeedd6f4c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:16.562517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:16.639378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f463fbd-1690-4f4b-8065-fbdbeedd6f4c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:16.675952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc2c2a-3a12-448e-b4b8-7ca0febbaefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:16.676558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc2c2a-3a12-448e-b4b8-7ca0febbaefc None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:16.676840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc2c2a-3a12-448e-b4b8-7ca0febbaefc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:16.677116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc2c2a-3a12-448e-b4b8-7ca0febbaefc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:16.677802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc2c2a-3a12-448e-b4b8-7ca0febbaefc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:16.679176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc2c2a-3a12-448e-b4b8-7ca0febbaefc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:16.761757 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 510/1020] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:31:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:17.544776 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c8c10-86cb-45de-99d0-a59c4ec2b2ae None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:17.545826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c8c10-86cb-45de-99d0-a59c4ec2b2ae None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:17.546341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c8c10-86cb-45de-99d0-a59c4ec2b2ae None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:17.548178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 511/1021] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:17.553944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acfb11-e508-4b60-86bf-915aefcbc481 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:17.555351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acfb11-e508-4b60-86bf-915aefcbc481 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:17.555835 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acfb11-e508-4b60-86bf-915aefcbc481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:17.579324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-96acfb11-e508-4b60-86bf-915aefcbc481 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:17.580574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-96acfb11-e508-4b60-86bf-915aefcbc481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:17.662047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 511/1022] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:17.958416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c8c10-86cb-45de-99d0-a59c4ec2b2ae None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:17.970386 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae0b4d-e2d0-40d2-9cba-481b00929381 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:17.970784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae0b4d-e2d0-40d2-9cba-481b00929381 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:17.971057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae0b4d-e2d0-40d2-9cba-481b00929381 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:17.972400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 512/1023] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:17.977617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f460-42e3-4fc6-944f-0414434ae93b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:17.977923 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f460-42e3-4fc6-944f-0414434ae93b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:17.978197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f460-42e3-4fc6-944f-0414434ae93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:17.985382 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:17.997452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a187f460-42e3-4fc6-944f-0414434ae93b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:17.998671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a187f460-42e3-4fc6-944f-0414434ae93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:18.072810 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c8c10-86cb-45de-99d0-a59c4ec2b2ae None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:18.078132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 513/1024] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:18.114045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4081446-9823-47c2-8a66-f46749348599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:18.114672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4081446-9823-47c2-8a66-f46749348599 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:18.114947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4081446-9823-47c2-8a66-f46749348599 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:18.115213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4081446-9823-47c2-8a66-f46749348599 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:18.116336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4081446-9823-47c2-8a66-f46749348599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:18.119001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4081446-9823-47c2-8a66-f46749348599 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:18.120578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 512/1025] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:18.126434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187f460-42e3-4fc6-944f-0414434ae93b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:18.227690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187f460-42e3-4fc6-944f-0414434ae93b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:18.261706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdc389-8e15-424e-a4b4-b4307f4fe0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:18.262334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc389-8e15-424e-a4b4-b4307f4fe0ff None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:18.262617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc389-8e15-424e-a4b4-b4307f4fe0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:18.262890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc389-8e15-424e-a4b4-b4307f4fe0ff None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:18.263543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bdc389-8e15-424e-a4b4-b4307f4fe0ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:18.264961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bdc389-8e15-424e-a4b4-b4307f4fe0ff None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:18.272885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:18.344453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 514/1026] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:31:18 2022] POST /identity/v3/users => generated 285 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:18.351161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4081446-9823-47c2-8a66-f46749348599 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:18.356838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:18.462983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4081446-9823-47c2-8a66-f46749348599 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:18.500018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acf7a-cf8f-4256-933f-28f5196270b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:18.500660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acf7a-cf8f-4256-933f-28f5196270b4 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:18.500932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acf7a-cf8f-4256-933f-28f5196270b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:18.501201 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acf7a-cf8f-4256-933f-28f5196270b4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:18.501821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acf7a-cf8f-4256-933f-28f5196270b4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:18.503210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acf7a-cf8f-4256-933f-28f5196270b4 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-844acf7a-cf8f-4256-933f-28f5196270b4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:18.512575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:18.513752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:18.514646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 513/1027] 10.222.0.42 () {56 vars in 1105 bytes} [Sat Sep 17 03:31:18 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:18.519737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdc389-8e15-424e-a4b4-b4307f4fe0ff None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:18.627587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdc389-8e15-424e-a4b4-b4307f4fe0ff None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:18.663746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7afb9-8a49-482a-975e-58d04cbed531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:18.665230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7afb9-8a49-482a-975e-58d04cbed531 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:18.665739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7afb9-8a49-482a-975e-58d04cbed531 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:18.666034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7afb9-8a49-482a-975e-58d04cbed531 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:18.666677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7afb9-8a49-482a-975e-58d04cbed531 None admin] RBAC: Adding query filter params (name=image) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:18.666981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7afb9-8a49-482a-975e-58d04cbed531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:18.668352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7afb9-8a49-482a-975e-58d04cbed531 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:18.678836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 515/1028] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:18.680752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acf7a-cf8f-4256-933f-28f5196270b4 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:18.752057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:18.787561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acf7a-cf8f-4256-933f-28f5196270b4 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:18.829252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb414a-d91e-4577-a275-8720c421dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:18.829885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb414a-d91e-4577-a275-8720c421dc52 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:18.830164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb414a-d91e-4577-a275-8720c421dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:18.830434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb414a-d91e-4577-a275-8720c421dc52 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:18.831063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb414a-d91e-4577-a275-8720c421dc52 None admin] RBAC: Adding query filter params (type=image) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:18.831363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb414a-d91e-4577-a275-8720c421dc52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:18.832726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb414a-d91e-4577-a275-8720c421dc52 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:18.842938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 514/1029] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:18.844458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7afb9-8a49-482a-975e-58d04cbed531 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:18.906358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:18.953007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7afb9-8a49-482a-975e-58d04cbed531 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:18.990989 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb82564-0622-458b-b1d6-66fdcc90d325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:18.991639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82564-0622-458b-b1d6-66fdcc90d325 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:18.991906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82564-0622-458b-b1d6-66fdcc90d325 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:18.992775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82564-0622-458b-b1d6-66fdcc90d325 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:18.993876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb82564-0622-458b-b1d6-66fdcc90d325 None admin] RBAC: Adding query filter params (interface=public, service_id=f5277a149e5d4a9ea4b6b1a8e5b8d398, region_id=RegionOne) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:18.994197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb82564-0622-458b-b1d6-66fdcc90d325 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:18.995593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb82564-0622-458b-b1d6-66fdcc90d325 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:19.005996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 516/1030] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Sep 17 03:31:18 2022] GET /identity/v3/endpoints?service_id=f5277a149e5d4a9ea4b6b1a8e5b8d398&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:19.007291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb414a-d91e-4577-a275-8720c421dc52 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:19.087565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b5b78-20ec-464f-af10-5fe77b505a42 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:19.087918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b5b78-20ec-464f-af10-5fe77b505a42 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:19.088143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b5b78-20ec-464f-af10-5fe77b505a42 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:19.089973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 517/1031] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:31:19.094964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13ba2-cde3-4def-917e-daaf90e4e08c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:19.095264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13ba2-cde3-4def-917e-daaf90e4e08c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:19.095546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13ba2-cde3-4def-917e-daaf90e4e08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:19.112567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb414a-d91e-4577-a275-8720c421dc52 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:19.114244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f4a13ba2-cde3-4def-917e-daaf90e4e08c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:19.115440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f4a13ba2-cde3-4def-917e-daaf90e4e08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:19.144641 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:19.151040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9ea17-4b53-4a85-8697-2a1174abfc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:19.155611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9ea17-4b53-4a85-8697-2a1174abfc64 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:19.155611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9ea17-4b53-4a85-8697-2a1174abfc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:19.155611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9ea17-4b53-4a85-8697-2a1174abfc64 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:19.155611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9ea17-4b53-4a85-8697-2a1174abfc64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:19.155611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9ea17-4b53-4a85-8697-2a1174abfc64 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:19.169851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 515/1032] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Sep 17 03:31:19 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:31:19.198429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 518/1033] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:19.472474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9ea17-4b53-4a85-8697-2a1174abfc64 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:19.575683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9ea17-4b53-4a85-8697-2a1174abfc64 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:19.609614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a4f5a-4587-427d-b693-a5c38ae4b208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:19.610267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a4f5a-4587-427d-b693-a5c38ae4b208 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:19.610545 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a4f5a-4587-427d-b693-a5c38ae4b208 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:19.610815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a4f5a-4587-427d-b693-a5c38ae4b208 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:19.623219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a4f5a-4587-427d-b693-a5c38ae4b208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-594a4f5a-4587-427d-b693-a5c38ae4b208 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:19.628199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:19.629170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:19.630094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 516/1034] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Sep 17 03:31:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:19.637594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a13ba2-cde3-4def-917e-daaf90e4e08c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:19.692899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:19.745454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a13ba2-cde3-4def-917e-daaf90e4e08c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:19.779874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03395581-b38f-45f4-918e-77216c0daaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:19.780489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03395581-b38f-45f4-918e-77216c0daaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:19.780768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03395581-b38f-45f4-918e-77216c0daaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:19.781039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03395581-b38f-45f4-918e-77216c0daaf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:19.781709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03395581-b38f-45f4-918e-77216c0daaf4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:19.782041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03395581-b38f-45f4-918e-77216c0daaf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:19.783402 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03395581-b38f-45f4-918e-77216c0daaf4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:19.792330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 519/1035] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Sep 17 03:31:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:19.804998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a4f5a-4587-427d-b693-a5c38ae4b208 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:19.861361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:19.905771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a4f5a-4587-427d-b693-a5c38ae4b208 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:19.939105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d924e2d-9f6a-467b-afce-a83f67b88cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:19.939729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d924e2d-9f6a-467b-afce-a83f67b88cfb None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:19.939999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d924e2d-9f6a-467b-afce-a83f67b88cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:19.940264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d924e2d-9f6a-467b-afce-a83f67b88cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:19.950944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d924e2d-9f6a-467b-afce-a83f67b88cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:19.952375 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d924e2d-9f6a-467b-afce-a83f67b88cfb None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-7d924e2d-9f6a-467b-afce-a83f67b88cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:19.966512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:19.967517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:19.973581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:19.973581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:19.973581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:19.973581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 517/1036] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:19.973866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03395581-b38f-45f4-918e-77216c0daaf4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:20.082975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03395581-b38f-45f4-918e-77216c0daaf4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:20.094364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:20.117631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063bed2-03a0-4feb-b147-5139894ab2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:20.118271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063bed2-03a0-4feb-b147-5139894ab2ba None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:20.118548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063bed2-03a0-4feb-b147-5139894ab2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:20.118858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063bed2-03a0-4feb-b147-5139894ab2ba None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:20.119449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063bed2-03a0-4feb-b147-5139894ab2ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:20.119748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063bed2-03a0-4feb-b147-5139894ab2ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:20.121096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063bed2-03a0-4feb-b147-5139894ab2ba None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:20.133676 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 520/1037] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:20.139182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d924e2d-9f6a-467b-afce-a83f67b88cfb None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:20.239125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d924e2d-9f6a-467b-afce-a83f67b88cfb None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:20.247973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:20.272267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7038d-c8d4-4e6b-a66f-f37fefab1100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:20.273111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7038d-c8d4-4e6b-a66f-f37fefab1100 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:20.273940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7038d-c8d4-4e6b-a66f-f37fefab1100 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:20.274392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7038d-c8d4-4e6b-a66f-f37fefab1100 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:20.282074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7038d-c8d4-4e6b-a66f-f37fefab1100 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:20.283500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7038d-c8d4-4e6b-a66f-f37fefab1100 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-5bb7038d-c8d4-4e6b-a66f-f37fefab1100 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:31:20.294823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:20.295786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:20.296770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:20.296770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:20.296770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:31:20.296770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:31:20.296770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:31:20.296770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:31:20.296770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:31:20.296770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:31:20.296770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:31:20.296770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:20.300793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 518/1038] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Sep 17 03:31:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:20.301806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063bed2-03a0-4feb-b147-5139894ab2ba None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:20.406348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063bed2-03a0-4feb-b147-5139894ab2ba None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:20.441466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6fa9b-0df6-4a42-aba0-61a51815dc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:20.442106 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6fa9b-0df6-4a42-aba0-61a51815dc10 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:20.442386 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6fa9b-0df6-4a42-aba0-61a51815dc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:20.442656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6fa9b-0df6-4a42-aba0-61a51815dc10 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:20.443377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6fa9b-0df6-4a42-aba0-61a51815dc10 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:20.443680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6fa9b-0df6-4a42-aba0-61a51815dc10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:20.445092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6fa9b-0df6-4a42-aba0-61a51815dc10 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:20.462920 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 521/1039] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Sep 17 03:31:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:20.468861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7038d-c8d4-4e6b-a66f-f37fefab1100 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:20.576905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7038d-c8d4-4e6b-a66f-f37fefab1100 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:20.596204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96bb15-785f-430d-a521-89e8953bc3f0 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:20.596495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96bb15-785f-430d-a521-89e8953bc3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:20.596776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96bb15-785f-430d-a521-89e8953bc3f0 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:20.598281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 522/1040] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:31:20.603792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db9dd4-e7ee-4e26-9d00-281c53e3cd76 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:20.604078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db9dd4-e7ee-4e26-9d00-281c53e3cd76 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:20.604359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db9dd4-e7ee-4e26-9d00-281c53e3cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:20.613957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc6b2b-f9e5-454c-9483-35cb48a20be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:20.614564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc6b2b-f9e5-454c-9483-35cb48a20be3 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:20.614843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc6b2b-f9e5-454c-9483-35cb48a20be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:20.615119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc6b2b-f9e5-454c-9483-35cb48a20be3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:20.622714 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-85db9dd4-e7ee-4e26-9d00-281c53e3cd76 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:20.623904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-85db9dd4-e7ee-4e26-9d00-281c53e3cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:20.648694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:20.650629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:20.690947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc6b2b-f9e5-454c-9483-35cb48a20be3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:20.692314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc6b2b-f9e5-454c-9483-35cb48a20be3 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:20.696350 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 523/1041] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:20.792164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 519/1042] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:31:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:31:20.801541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db9dd4-e7ee-4e26-9d00-281c53e3cd76 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:20.900664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db9dd4-e7ee-4e26-9d00-281c53e3cd76 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:20.936527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ce3e4-e29d-4a3a-944f-e57c3f57cc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:20.937153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ce3e4-e29d-4a3a-944f-e57c3f57cc39 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:20.937435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ce3e4-e29d-4a3a-944f-e57c3f57cc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:20.937708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ce3e4-e29d-4a3a-944f-e57c3f57cc39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:20.948387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ce3e4-e29d-4a3a-944f-e57c3f57cc39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:20.950115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ce3e4-e29d-4a3a-944f-e57c3f57cc39 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-592ce3e4-e29d-4a3a-944f-e57c3f57cc39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:20.965149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:20.966359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:20.967309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:20.967309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:20.967309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:20.967516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc6b2b-f9e5-454c-9483-35cb48a20be3 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:20.968357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 524/1043] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:20.977847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ce3e4-e29d-4a3a-944f-e57c3f57cc39 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:20.990266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:21.052855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:21.080710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ce3e4-e29d-4a3a-944f-e57c3f57cc39 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:21.093485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc6b2b-f9e5-454c-9483-35cb48a20be3 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:21.116351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd45a83-cc1c-4521-9a83-d3f139abccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:21.116955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd45a83-cc1c-4521-9a83-d3f139abccba None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.117265 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd45a83-cc1c-4521-9a83-d3f139abccba None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.117541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd45a83-cc1c-4521-9a83-d3f139abccba None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.118218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd45a83-cc1c-4521-9a83-d3f139abccba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:21.118525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd45a83-cc1c-4521-9a83-d3f139abccba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:21.119890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd45a83-cc1c-4521-9a83-d3f139abccba None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:21.128763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d8e2f-4029-467c-87d7-2bda57bffd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:21.129393 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d8e2f-4029-467c-87d7-2bda57bffd00 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.129664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d8e2f-4029-467c-87d7-2bda57bffd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.129964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d8e2f-4029-467c-87d7-2bda57bffd00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.138077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 525/1044] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:21.139408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd45a83-cc1c-4521-9a83-d3f139abccba None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:21.141163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d8e2f-4029-467c-87d7-2bda57bffd00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-374d8e2f-4029-467c-87d7-2bda57bffd00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:21.145968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:21.147076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:21.147965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 520/1045] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:31:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:21.154581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d8e2f-4029-467c-87d7-2bda57bffd00 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:21.220505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:21.256507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd45a83-cc1c-4521-9a83-d3f139abccba None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:21.261669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d8e2f-4029-467c-87d7-2bda57bffd00 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:21.293427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:21.296228 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d369d4-29a9-4679-a6df-dec067981b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:21.296853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d369d4-29a9-4679-a6df-dec067981b43 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.297130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d369d4-29a9-4679-a6df-dec067981b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.297405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d369d4-29a9-4679-a6df-dec067981b43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.301872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274154c-146f-418c-9de8-0c8590393961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:21.304555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274154c-146f-418c-9de8-0c8590393961 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.304555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274154c-146f-418c-9de8-0c8590393961 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.304555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274154c-146f-418c-9de8-0c8590393961 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.304555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274154c-146f-418c-9de8-0c8590393961 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:21.304555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274154c-146f-418c-9de8-0c8590393961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:21.304899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d369d4-29a9-4679-a6df-dec067981b43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:21.305672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d369d4-29a9-4679-a6df-dec067981b43 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:21.305803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274154c-146f-418c-9de8-0c8590393961 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-68d369d4-29a9-4679-a6df-dec067981b43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:31:21.317232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:21.318227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:21.319194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:31:21.319194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:31:21.319194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:31:21.319194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:31:21.319194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:31:21.319194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:31:21.319194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:21.319194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:21.319652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 521/1046] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:31:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:21.321529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274154c-146f-418c-9de8-0c8590393961 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:21.322093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 526/1047] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:31:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:21.325262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d369d4-29a9-4679-a6df-dec067981b43 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:21.422258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274154c-146f-418c-9de8-0c8590393961 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:21.432381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d369d4-29a9-4679-a6df-dec067981b43 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:21.456484 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196d75e-b71d-456a-88a7-bd9beb308a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:21.457106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196d75e-b71d-456a-88a7-bd9beb308a2a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.457387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196d75e-b71d-456a-88a7-bd9beb308a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.457662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196d75e-b71d-456a-88a7-bd9beb308a2a None admin] PATH_INFO: `/v3/users/2def5a391d6b474cbc7027b144a818b5` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.460061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:21.465647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53903ce9-3369-48c3-a14d-b7a531223a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:21.466276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53903ce9-3369-48c3-a14d-b7a531223a31 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.466558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53903ce9-3369-48c3-a14d-b7a531223a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.466832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53903ce9-3369-48c3-a14d-b7a531223a31 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.467557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53903ce9-3369-48c3-a14d-b7a531223a31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:21.467859 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53903ce9-3369-48c3-a14d-b7a531223a31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:21.469283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53903ce9-3369-48c3-a14d-b7a531223a31 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:21.471300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196d75e-b71d-456a-88a7-bd9beb308a2a None admin] RBAC: Authorizing `identity:get_user(user_id=2def5a391d6b474cbc7027b144a818b5)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:21.472658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196d75e-b71d-456a-88a7-bd9beb308a2a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:21.474380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 522/1048] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:21 2022] GET /identity/v3/users/2def5a391d6b474cbc7027b144a818b5 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:21.475602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bb741-109f-44d5-be32-1f7c0f818c36 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.476288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bb741-109f-44d5-be32-1f7c0f818c36 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.476288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bb741-109f-44d5-be32-1f7c0f818c36 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.477487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 523/1049] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:21.480089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bb741-109f-44d5-be32-1f7c0f818c36 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:21.513493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 527/1050] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:31:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:21.514642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953643b-200c-4230-acfd-799407abad84 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.514964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953643b-200c-4230-acfd-799407abad84 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.515276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953643b-200c-4230-acfd-799407abad84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.532541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3953643b-200c-4230-acfd-799407abad84 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:21.533740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3953643b-200c-4230-acfd-799407abad84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:21.533914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:21.586882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bb741-109f-44d5-be32-1f7c0f818c36 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:21.602693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 528/1051] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:21.603531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953643b-200c-4230-acfd-799407abad84 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:21.624238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d786e-76f4-4736-ac63-dc459c6b4197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:21.624846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d786e-76f4-4736-ac63-dc459c6b4197 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.625124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d786e-76f4-4736-ac63-dc459c6b4197 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.625396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d786e-76f4-4736-ac63-dc459c6b4197 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.699002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d786e-76f4-4736-ac63-dc459c6b4197 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:21.700804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d786e-76f4-4736-ac63-dc459c6b4197 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:21.722567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953643b-200c-4230-acfd-799407abad84 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:21.742934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:21.753548 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:21.757441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441de94-3492-414e-a001-d005f91ad7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:21.759153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441de94-3492-414e-a001-d005f91ad7cb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.759586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441de94-3492-414e-a001-d005f91ad7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.759871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441de94-3492-414e-a001-d005f91ad7cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.760595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441de94-3492-414e-a001-d005f91ad7cb None admin] RBAC: Adding query filter params (role.id=e88868b1e3fc44e6aee8a63eb09ddd9b, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=9901608b62344115aa561d00bfe88c1c) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:21.760909 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441de94-3492-414e-a001-d005f91ad7cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:21.762357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441de94-3492-414e-a001-d005f91ad7cb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:21.769753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 529/1052] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:31:21 2022] GET /identity/v3/role_assignments?user.id=9901608b62344115aa561d00bfe88c1c&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e88868b1e3fc44e6aee8a63eb09ddd9b => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:21.822511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 524/1053] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:31:21 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 03:31:21.825719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441de94-3492-414e-a001-d005f91ad7cb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:21.881923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d786e-76f4-4736-ac63-dc459c6b4197 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:21.940201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441de94-3492-414e-a001-d005f91ad7cb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:21.978584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b073e-a3c3-4ecf-9a04-9b94b0936e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:21.979131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b073e-a3c3-4ecf-9a04-9b94b0936e90 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:21.979413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b073e-a3c3-4ecf-9a04-9b94b0936e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:21.979745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b073e-a3c3-4ecf-9a04-9b94b0936e90 None admin] PATH_INFO: `/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:21.980897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b073e-a3c3-4ecf-9a04-9b94b0936e90 None admin] RBAC: Authorizing `identity:get_project(project_id=3b4fb3599b6c4207a5d8ed34e635adbd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:21.982356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b073e-a3c3-4ecf-9a04-9b94b0936e90 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:21.983991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 530/1054] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:31:21 2022] GET /identity/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:21.989965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b073e-a3c3-4ecf-9a04-9b94b0936e90 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:21.996198 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d786e-76f4-4736-ac63-dc459c6b4197 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:22.031799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea3d30-30a6-4882-94c2-d1d42a36813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:22.032433 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea3d30-30a6-4882-94c2-d1d42a36813f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:22.032710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea3d30-30a6-4882-94c2-d1d42a36813f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:22.032983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea3d30-30a6-4882-94c2-d1d42a36813f None admin] PATH_INFO: `/v3/services/image` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:22.033564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea3d30-30a6-4882-94c2-d1d42a36813f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:22.034963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea3d30-30a6-4882-94c2-d1d42a36813f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-77ea3d30-30a6-4882-94c2-d1d42a36813f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:22.044397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:22.045434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 03:31:22.046343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:22.048908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:22.050510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 525/1055] 10.222.0.42 () {56 vars in 1105 bytes} [Sat Sep 17 03:31:21 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:22.051364 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea3d30-30a6-4882-94c2-d1d42a36813f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:22.066754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:22.096636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b073e-a3c3-4ecf-9a04-9b94b0936e90 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:22.135867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32320098-f48d-4f8f-a146-74d910faf620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:22.136529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32320098-f48d-4f8f-a146-74d910faf620 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:22.136838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32320098-f48d-4f8f-a146-74d910faf620 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:22.137092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32320098-f48d-4f8f-a146-74d910faf620 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:22.137854 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32320098-f48d-4f8f-a146-74d910faf620 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=3b4fb3599b6c4207a5d8ed34e635adbd, user.id=2def5a391d6b474cbc7027b144a818b5) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:22.138167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32320098-f48d-4f8f-a146-74d910faf620 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:22.139591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32320098-f48d-4f8f-a146-74d910faf620 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:22.157753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 531/1056] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:31:21 2022] GET /identity/v3/role_assignments?user.id=2def5a391d6b474cbc7027b144a818b5&scope.project.id=3b4fb3599b6c4207a5d8ed34e635adbd&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:22.157971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea3d30-30a6-4882-94c2-d1d42a36813f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:22.196767 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ce79c-709e-4a6b-a749-a68de22bdf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:22.197391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ce79c-709e-4a6b-a749-a68de22bdf75 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:22.197667 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ce79c-709e-4a6b-a749-a68de22bdf75 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:22.197967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ce79c-709e-4a6b-a749-a68de22bdf75 None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:22.198621 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ce79c-709e-4a6b-a749-a68de22bdf75 None admin] RBAC: Adding query filter params (name=image) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:22.199180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ce79c-709e-4a6b-a749-a68de22bdf75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:22.200586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ce79c-709e-4a6b-a749-a68de22bdf75 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:22.212056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 526/1057] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:22.212980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32320098-f48d-4f8f-a146-74d910faf620 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:22.323208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32320098-f48d-4f8f-a146-74d910faf620 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:22.342987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:22.361107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff06c61-c54f-40ac-8545-14c6295473d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:22.361712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06c61-c54f-40ac-8545-14c6295473d5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:22.362020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06c61-c54f-40ac-8545-14c6295473d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:22.362294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06c61-c54f-40ac-8545-14c6295473d5 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:22.362918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06c61-c54f-40ac-8545-14c6295473d5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:22.363218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06c61-c54f-40ac-8545-14c6295473d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:22.364630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06c61-c54f-40ac-8545-14c6295473d5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:22.370393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 532/1058] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:22.375909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ce79c-709e-4a6b-a749-a68de22bdf75 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:22.457910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:22.481567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ce79c-709e-4a6b-a749-a68de22bdf75 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:22.517617 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08272b-fe98-4663-b389-7c575757306e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:22.518329 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08272b-fe98-4663-b389-7c575757306e None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:22.518608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08272b-fe98-4663-b389-7c575757306e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:22.518879 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08272b-fe98-4663-b389-7c575757306e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:22.519551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08272b-fe98-4663-b389-7c575757306e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:22.520914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08272b-fe98-4663-b389-7c575757306e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:22.620814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 527/1059] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:31:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:23.933215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba7bab-c86b-4e31-a9b9-9c493a5b544f None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:23.933898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba7bab-c86b-4e31-a9b9-9c493a5b544f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:23.933898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba7bab-c86b-4e31-a9b9-9c493a5b544f None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:23.935292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 533/1060] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:23.940186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129a7b2-a2b8-452e-a3ad-c1e00960d703 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:23.940567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129a7b2-a2b8-452e-a3ad-c1e00960d703 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:23.940729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129a7b2-a2b8-452e-a3ad-c1e00960d703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:23.959138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d129a7b2-a2b8-452e-a3ad-c1e00960d703 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:23.960302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d129a7b2-a2b8-452e-a3ad-c1e00960d703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:24.033240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 528/1061] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:24.299956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba7bab-c86b-4e31-a9b9-9c493a5b544f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:24.317058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ae785-d6ad-41b8-99a1-cbd20741cdb3 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:24.317343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ae785-d6ad-41b8-99a1-cbd20741cdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:24.317608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ae785-d6ad-41b8-99a1-cbd20741cdb3 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:24.319258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 529/1062] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:31:24.325027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06eed5-1ef3-4470-92d3-37b44d0199ce None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:24.325308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06eed5-1ef3-4470-92d3-37b44d0199ce None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:24.325576 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06eed5-1ef3-4470-92d3-37b44d0199ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:24.344273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-dc06eed5-1ef3-4470-92d3-37b44d0199ce None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:24.345446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-dc06eed5-1ef3-4470-92d3-37b44d0199ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:24.405763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:24.410180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba7bab-c86b-4e31-a9b9-9c493a5b544f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:24.420972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 530/1063] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:24.446939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14d027-ccfc-43e6-abe4-0eb1dfb291ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:24.447556 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14d027-ccfc-43e6-abe4-0eb1dfb291ab None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:24.447834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14d027-ccfc-43e6-abe4-0eb1dfb291ab None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:24.448107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14d027-ccfc-43e6-abe4-0eb1dfb291ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:24.459210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14d027-ccfc-43e6-abe4-0eb1dfb291ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-5d14d027-ccfc-43e6-abe4-0eb1dfb291ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:24.464824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:24.465775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:31:24.466735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:24.470066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 534/1064] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Sep 17 03:31:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:24.471992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06eed5-1ef3-4470-92d3-37b44d0199ce None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:24.572682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06eed5-1ef3-4470-92d3-37b44d0199ce None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:24.606703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725bc80-0924-4bb2-9e07-aef54325d6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:24.607300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725bc80-0924-4bb2-9e07-aef54325d6be None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:24.607570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725bc80-0924-4bb2-9e07-aef54325d6be None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:24.607832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725bc80-0924-4bb2-9e07-aef54325d6be None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:24.608479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725bc80-0924-4bb2-9e07-aef54325d6be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:24.608773 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725bc80-0924-4bb2-9e07-aef54325d6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:24.610150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725bc80-0924-4bb2-9e07-aef54325d6be None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:24.623213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 531/1065] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Sep 17 03:31:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:24.624643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14d027-ccfc-43e6-abe4-0eb1dfb291ab None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:24.668767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:24.702337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725bc80-0924-4bb2-9e07-aef54325d6be None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:24.738396 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14d027-ccfc-43e6-abe4-0eb1dfb291ab None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:24.759102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:24.776944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f99eb0-f381-4030-8cf5-2341f3521439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:24.777673 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f99eb0-f381-4030-8cf5-2341f3521439 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:24.777982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f99eb0-f381-4030-8cf5-2341f3521439 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:24.778254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f99eb0-f381-4030-8cf5-2341f3521439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:24.788656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f99eb0-f381-4030-8cf5-2341f3521439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:24.790489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f99eb0-f381-4030-8cf5-2341f3521439 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-96f99eb0-f381-4030-8cf5-2341f3521439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:24.806001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:24.806975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:24.808178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:24.808178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:24.808178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:24.808178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 535/1066] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:24.812665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f99eb0-f381-4030-8cf5-2341f3521439 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:24.822145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725bc80-0924-4bb2-9e07-aef54325d6be None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:24.857081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea640267-20c8-4384-811c-7a66d99db296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:24.857618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea640267-20c8-4384-811c-7a66d99db296 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:24.857922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea640267-20c8-4384-811c-7a66d99db296 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:24.858193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea640267-20c8-4384-811c-7a66d99db296 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:24.868236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea640267-20c8-4384-811c-7a66d99db296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-ea640267-20c8-4384-811c-7a66d99db296 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:24.872918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:24.873903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:24.874794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 532/1067] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:31:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:24.875535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23990dd5-6654-4e4f-ad5a-a1edcb0ce866 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:24.875843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23990dd5-6654-4e4f-ad5a-a1edcb0ce866 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:24.875946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23990dd5-6654-4e4f-ad5a-a1edcb0ce866 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:24.877157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 533/1068] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:24.879911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23990dd5-6654-4e4f-ad5a-a1edcb0ce866 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:24.907588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:24.916774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f99eb0-f381-4030-8cf5-2341f3521439 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:24.952223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69329b91-349d-47ff-b53d-0a7bcecffc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:24.952867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69329b91-349d-47ff-b53d-0a7bcecffc11 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:24.953086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69329b91-349d-47ff-b53d-0a7bcecffc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:24.953360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69329b91-349d-47ff-b53d-0a7bcecffc11 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:24.954024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69329b91-349d-47ff-b53d-0a7bcecffc11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:24.954326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69329b91-349d-47ff-b53d-0a7bcecffc11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:24.955677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69329b91-349d-47ff-b53d-0a7bcecffc11 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:24.966545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 536/1069] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:24.967694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05984dd2-f7db-4d07-8911-1a02b9684c00 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:24.967975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05984dd2-f7db-4d07-8911-1a02b9684c00 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:24.968249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05984dd2-f7db-4d07-8911-1a02b9684c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:24.987800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-05984dd2-f7db-4d07-8911-1a02b9684c00 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:24.989039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-05984dd2-f7db-4d07-8911-1a02b9684c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:24.993740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23990dd5-6654-4e4f-ad5a-a1edcb0ce866 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.004831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:25.027690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458714c-fcfd-4a12-bdb0-ef161b70f671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:25.029874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458714c-fcfd-4a12-bdb0-ef161b70f671 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:25.029874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458714c-fcfd-4a12-bdb0-ef161b70f671 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:25.029874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458714c-fcfd-4a12-bdb0-ef161b70f671 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:25.029874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458714c-fcfd-4a12-bdb0-ef161b70f671 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:25.029874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458714c-fcfd-4a12-bdb0-ef161b70f671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:25.031137 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458714c-fcfd-4a12-bdb0-ef161b70f671 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:25.040597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 534/1070] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:31:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:25.041814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458714c-fcfd-4a12-bdb0-ef161b70f671 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:25.061203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 537/1071] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:25.061930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05984dd2-f7db-4d07-8911-1a02b9684c00 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:25.148261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:25.148460 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458714c-fcfd-4a12-bdb0-ef161b70f671 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.166942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05984dd2-f7db-4d07-8911-1a02b9684c00 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.186394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174b936-45d9-414f-9a5e-d878cd2b079f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:25.187018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174b936-45d9-414f-9a5e-d878cd2b079f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:25.187296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174b936-45d9-414f-9a5e-d878cd2b079f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:25.187569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174b936-45d9-414f-9a5e-d878cd2b079f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:25.194488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174b936-45d9-414f-9a5e-d878cd2b079f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:25.195975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174b936-45d9-414f-9a5e-d878cd2b079f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:25.202760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e869f4-db54-4002-9306-5a9f217d3bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:25.203378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e869f4-db54-4002-9306-5a9f217d3bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:25.203653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e869f4-db54-4002-9306-5a9f217d3bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:25.203920 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e869f4-db54-4002-9306-5a9f217d3bf8 None admin] PATH_INFO: `/v3/users/2def5a391d6b474cbc7027b144a818b5` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:25.205024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e869f4-db54-4002-9306-5a9f217d3bf8 None admin] RBAC: Authorizing `identity:get_user(user_id=2def5a391d6b474cbc7027b144a818b5)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:25.206448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e869f4-db54-4002-9306-5a9f217d3bf8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d174b936-45d9-414f-9a5e-d878cd2b079f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:31:25.207596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:25.208575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:25.209706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:25.209706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:25.209706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:31:25.209706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:31:25.209706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:31:25.209706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:31:25.209706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:31:25.209706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:31:25.209706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:31:25.209706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:25.210354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 538/1072] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:25 2022] GET /identity/v3/users/2def5a391d6b474cbc7027b144a818b5 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:25.211416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 535/1073] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Sep 17 03:31:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:25.214139 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e869f4-db54-4002-9306-5a9f217d3bf8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:25.216549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174b936-45d9-414f-9a5e-d878cd2b079f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:25.250990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:25.319833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e869f4-db54-4002-9306-5a9f217d3bf8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.323114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174b936-45d9-414f-9a5e-d878cd2b079f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.355474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f68de-2da8-42a9-9088-3ad39f24567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:25.355740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e834-f197-446a-a945-7ebf5d9d9158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:25.356251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f68de-2da8-42a9-9088-3ad39f24567c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:25.356359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e834-f197-446a-a945-7ebf5d9d9158 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:25.356608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e834-f197-446a-a945-7ebf5d9d9158 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:25.356707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f68de-2da8-42a9-9088-3ad39f24567c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:25.356879 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e834-f197-446a-a945-7ebf5d9d9158 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:25.357087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f68de-2da8-42a9-9088-3ad39f24567c None admin] PATH_INFO: `/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:25.357600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16e834-f197-446a-a945-7ebf5d9d9158 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:25.357937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16e834-f197-446a-a945-7ebf5d9d9158 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:25.358274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f68de-2da8-42a9-9088-3ad39f24567c None admin] RBAC: Authorizing `identity:get_project(project_id=3b4fb3599b6c4207a5d8ed34e635adbd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:25.359361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16e834-f197-446a-a945-7ebf5d9d9158 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:25.359678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f68de-2da8-42a9-9088-3ad39f24567c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:25.361277 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 539/1074] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:31:25 2022] GET /identity/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:25.362044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f68de-2da8-42a9-9088-3ad39f24567c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:25.380092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:25.383895 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 536/1075] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Sep 17 03:31:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:25.384696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e834-f197-446a-a945-7ebf5d9d9158 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:25.461460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f68de-2da8-42a9-9088-3ad39f24567c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.479266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:25.488723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e834-f197-446a-a945-7ebf5d9d9158 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.495872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fada4a-7af1-41b4-85c8-2900484d860d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:25.496480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fada4a-7af1-41b4-85c8-2900484d860d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:25.496761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fada4a-7af1-41b4-85c8-2900484d860d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:25.497035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fada4a-7af1-41b4-85c8-2900484d860d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:25.523675 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada3b41-785f-4958-9e18-3b16590aa323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:25.524232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3b41-785f-4958-9e18-3b16590aa323 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:25.524511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3b41-785f-4958-9e18-3b16590aa323 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:25.524793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3b41-785f-4958-9e18-3b16590aa323 None admin] PATH_INFO: `/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd/users/2def5a391d6b474cbc7027b144a818b5/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:25.526753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada3b41-785f-4958-9e18-3b16590aa323 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4fb3599b6c4207a5d8ed34e635adbd, user_id=2def5a391d6b474cbc7027b144a818b5, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:25.528187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada3b41-785f-4958-9e18-3b16590aa323 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:25.563274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fada4a-7af1-41b4-85c8-2900484d860d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:25.564755 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fada4a-7af1-41b4-85c8-2900484d860d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:25.582381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:25.599542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 537/1076] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:31:25 2022] PUT /identity/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd/users/2def5a391d6b474cbc7027b144a818b5/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:31:25.608034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada3b41-785f-4958-9e18-3b16590aa323 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:25.677873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 540/1077] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:31:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:31:25.684735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fada4a-7af1-41b4-85c8-2900484d860d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:25.724629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada3b41-785f-4958-9e18-3b16590aa323 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.764036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878aea1-49af-4445-bc9d-90da9f2b71ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:25.764747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878aea1-49af-4445-bc9d-90da9f2b71ab None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:25.765026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878aea1-49af-4445-bc9d-90da9f2b71ab None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:25.765295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878aea1-49af-4445-bc9d-90da9f2b71ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:25.782284 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878aea1-49af-4445-bc9d-90da9f2b71ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:25.784496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878aea1-49af-4445-bc9d-90da9f2b71ab None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:25.797863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fada4a-7af1-41b4-85c8-2900484d860d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-b878aea1-49af-4445-bc9d-90da9f2b71ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:25.808719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:25.810028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:25.811150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:25.811150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:25.811150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:25.813398 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 538/1078] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:25.818841 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878aea1-49af-4445-bc9d-90da9f2b71ab None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:25.822434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:25.823512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:25.843071 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593118b-efd3-484d-988f-e16ae5033f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:25.843769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593118b-efd3-484d-988f-e16ae5033f10 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:25.849297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593118b-efd3-484d-988f-e16ae5033f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:25.849297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593118b-efd3-484d-988f-e16ae5033f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:25.854746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593118b-efd3-484d-988f-e16ae5033f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:25.856252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593118b-efd3-484d-988f-e16ae5033f10 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-7593118b-efd3-484d-988f-e16ae5033f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:25.870745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:25.871740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:25.872682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:25.872682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:25.872682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:25.872682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 541/1079] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:25.877705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593118b-efd3-484d-988f-e16ae5033f10 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:25.933965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878aea1-49af-4445-bc9d-90da9f2b71ab None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.973093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173474c-a054-41c7-a195-aa2537e1af64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:25.973702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173474c-a054-41c7-a195-aa2537e1af64 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:25.974010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173474c-a054-41c7-a195-aa2537e1af64 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:25.974585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173474c-a054-41c7-a195-aa2537e1af64 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:25.974974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173474c-a054-41c7-a195-aa2537e1af64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:25.975226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173474c-a054-41c7-a195-aa2537e1af64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:25.977008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173474c-a054-41c7-a195-aa2537e1af64 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:25.986756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593118b-efd3-484d-988f-e16ae5033f10 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:25.989673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 539/1080] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:25.995635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173474c-a054-41c7-a195-aa2537e1af64 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:26.027379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e594e1b-d493-42ae-9fa1-dbf21b9f2691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:26.027977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594e1b-d493-42ae-9fa1-dbf21b9f2691 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:26.028254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594e1b-d493-42ae-9fa1-dbf21b9f2691 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:26.028524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594e1b-d493-42ae-9fa1-dbf21b9f2691 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:26.029155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e594e1b-d493-42ae-9fa1-dbf21b9f2691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:26.029453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e594e1b-d493-42ae-9fa1-dbf21b9f2691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:26.031061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e594e1b-d493-42ae-9fa1-dbf21b9f2691 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:26.044296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 542/1081] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:26.071576 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:26.116033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173474c-a054-41c7-a195-aa2537e1af64 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:26.154405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c13e67-c541-4fc1-9449-4bd6eff90120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:26.155045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c13e67-c541-4fc1-9449-4bd6eff90120 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:26.155319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c13e67-c541-4fc1-9449-4bd6eff90120 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:26.155586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c13e67-c541-4fc1-9449-4bd6eff90120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:26.163063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c13e67-c541-4fc1-9449-4bd6eff90120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:26.164672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c13e67-c541-4fc1-9449-4bd6eff90120 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-69c13e67-c541-4fc1-9449-4bd6eff90120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:31:26.176664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:26.177690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:26.178981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:31:26.178981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:31:26.178981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:31:26.178981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:31:26.178981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:31:26.178981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:31:26.178981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:26.178981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:26.182084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 540/1082] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:31:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:26.184393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e594e1b-d493-42ae-9fa1-dbf21b9f2691 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:26.216343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:26.291028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e594e1b-d493-42ae-9fa1-dbf21b9f2691 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:26.330034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ab58f-2b85-4e1c-8e4f-797e7c16c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:26.330648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ab58f-2b85-4e1c-8e4f-797e7c16c34d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:26.331013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ab58f-2b85-4e1c-8e4f-797e7c16c34d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:26.331203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ab58f-2b85-4e1c-8e4f-797e7c16c34d None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:26.331899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ab58f-2b85-4e1c-8e4f-797e7c16c34d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:26.332210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ab58f-2b85-4e1c-8e4f-797e7c16c34d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:26.333648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ab58f-2b85-4e1c-8e4f-797e7c16c34d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:26.345941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 543/1083] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:31:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:26.352486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c13e67-c541-4fc1-9449-4bd6eff90120 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:26.459408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c13e67-c541-4fc1-9449-4bd6eff90120 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:26.481533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:26.493965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7153c-169a-4582-9005-d60768120499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:26.494639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7153c-169a-4582-9005-d60768120499 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:26.494917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7153c-169a-4582-9005-d60768120499 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:26.495184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7153c-169a-4582-9005-d60768120499 None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/9901608b62344115aa561d00bfe88c1c/roles/e88868b1e3fc44e6aee8a63eb09ddd9b` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:26.497417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7153c-169a-4582-9005-d60768120499 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=9901608b62344115aa561d00bfe88c1c, role_id=e88868b1e3fc44e6aee8a63eb09ddd9b)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:26.498992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7153c-169a-4582-9005-d60768120499 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:26.548268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 541/1084] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:31:26 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/9901608b62344115aa561d00bfe88c1c/roles/e88868b1e3fc44e6aee8a63eb09ddd9b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:31:27.783389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56923479-3e01-4a93-8f3b-87fed0087ee9 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:27.783883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56923479-3e01-4a93-8f3b-87fed0087ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:27.783883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56923479-3e01-4a93-8f3b-87fed0087ee9 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:27.785206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 544/1085] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:27.790866 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149ea4d-edd3-4631-a0f5-3cfe86416d11 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:27.791352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149ea4d-edd3-4631-a0f5-3cfe86416d11 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:27.791450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149ea4d-edd3-4631-a0f5-3cfe86416d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:27.810222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6149ea4d-edd3-4631-a0f5-3cfe86416d11 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:27.811413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6149ea4d-edd3-4631-a0f5-3cfe86416d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:27.879151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 542/1086] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:28.169140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56923479-3e01-4a93-8f3b-87fed0087ee9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:28.261650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:28.271371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56923479-3e01-4a93-8f3b-87fed0087ee9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:28.307194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb518e-9be8-4760-acac-c39edc3e8c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:28.307814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb518e-9be8-4760-acac-c39edc3e8c80 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:28.308097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb518e-9be8-4760-acac-c39edc3e8c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:28.308374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb518e-9be8-4760-acac-c39edc3e8c80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:28.318321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb518e-9be8-4760-acac-c39edc3e8c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-86cb518e-9be8-4760-acac-c39edc3e8c80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:28.323617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:28.324998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:28.325918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 545/1087] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:31:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:28.331104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149ea4d-edd3-4631-a0f5-3cfe86416d11 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:28.430013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149ea4d-edd3-4631-a0f5-3cfe86416d11 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:28.442699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:28.464142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc20a9-6103-4691-80fc-ebeab72a4834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:28.464743 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc20a9-6103-4691-80fc-ebeab72a4834 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:28.465024 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc20a9-6103-4691-80fc-ebeab72a4834 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:28.465297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc20a9-6103-4691-80fc-ebeab72a4834 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:28.466009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc20a9-6103-4691-80fc-ebeab72a4834 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:28.466312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc20a9-6103-4691-80fc-ebeab72a4834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:28.467731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc20a9-6103-4691-80fc-ebeab72a4834 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:28.481846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 543/1088] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:31:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:28.483333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb518e-9be8-4760-acac-c39edc3e8c80 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:28.595315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb518e-9be8-4760-acac-c39edc3e8c80 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:28.631894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06147cb1-53ba-4f4d-bc0e-96255b6ca515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:28.633159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06147cb1-53ba-4f4d-bc0e-96255b6ca515 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:28.633504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06147cb1-53ba-4f4d-bc0e-96255b6ca515 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:28.633814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06147cb1-53ba-4f4d-bc0e-96255b6ca515 None admin] PATH_INFO: `/v3/users/2def5a391d6b474cbc7027b144a818b5` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:28.634948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06147cb1-53ba-4f4d-bc0e-96255b6ca515 None admin] RBAC: Authorizing `identity:get_user(user_id=2def5a391d6b474cbc7027b144a818b5)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:28.636365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06147cb1-53ba-4f4d-bc0e-96255b6ca515 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:28.637983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 546/1089] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:28 2022] GET /identity/v3/users/2def5a391d6b474cbc7027b144a818b5 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:28.643766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc20a9-6103-4691-80fc-ebeab72a4834 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:28.717680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab83dcb-1186-4576-930e-9e0b4871975d None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:28.717897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab83dcb-1186-4576-930e-9e0b4871975d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:28.718174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab83dcb-1186-4576-930e-9e0b4871975d None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:28.719521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 547/1090] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:31:28.724568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9d971-591d-4529-bfcd-72b3e34d666c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:28.724857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9d971-591d-4529-bfcd-72b3e34d666c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:28.725142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9d971-591d-4529-bfcd-72b3e34d666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:28.744125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b3b9d971-591d-4529-bfcd-72b3e34d666c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:28.745306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b3b9d971-591d-4529-bfcd-72b3e34d666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:28.747351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc20a9-6103-4691-80fc-ebeab72a4834 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:28.758336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:28.780080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838594a-9e4d-4c1d-acaa-7e210f41e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:28.780684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838594a-9e4d-4c1d-acaa-7e210f41e0af None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:28.781018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838594a-9e4d-4c1d-acaa-7e210f41e0af None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:28.781297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838594a-9e4d-4c1d-acaa-7e210f41e0af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:28.813640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 548/1091] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:28.828032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:28.853178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838594a-9e4d-4c1d-acaa-7e210f41e0af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:28.854639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838594a-9e4d-4c1d-acaa-7e210f41e0af None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:28.951889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 544/1092] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:31:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 03:31:28.957798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9d971-591d-4529-bfcd-72b3e34d666c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:29.055823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9d971-591d-4529-bfcd-72b3e34d666c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:29.087572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838594a-9e4d-4c1d-acaa-7e210f41e0af None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:29.088543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbd428-6b05-40f3-8cca-e4330153a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:29.089165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbd428-6b05-40f3-8cca-e4330153a92c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:29.089463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbd428-6b05-40f3-8cca-e4330153a92c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:29.089718 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbd428-6b05-40f3-8cca-e4330153a92c None admin] PATH_INFO: `/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:29.090883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbd428-6b05-40f3-8cca-e4330153a92c None admin] RBAC: Authorizing `identity:get_project(project_id=3b4fb3599b6c4207a5d8ed34e635adbd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:29.092307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbd428-6b05-40f3-8cca-e4330153a92c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:29.093954 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 549/1093] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:31:28 2022] GET /identity/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:29.099961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbd428-6b05-40f3-8cca-e4330153a92c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:29.118996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:29.163467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:29.188937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838594a-9e4d-4c1d-acaa-7e210f41e0af None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:29.197745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbd428-6b05-40f3-8cca-e4330153a92c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:29.224849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ad4ba-8f58-4041-9ce8-ab416a799470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:29.225468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ad4ba-8f58-4041-9ce8-ab416a799470 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:29.225768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ad4ba-8f58-4041-9ce8-ab416a799470 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:29.226044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ad4ba-8f58-4041-9ce8-ab416a799470 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:29.232469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a28f5-15dc-4ac1-a432-f5917d0002da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:29.233062 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a28f5-15dc-4ac1-a432-f5917d0002da None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:29.233343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a28f5-15dc-4ac1-a432-f5917d0002da None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:29.233618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a28f5-15dc-4ac1-a432-f5917d0002da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:29.234370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a28f5-15dc-4ac1-a432-f5917d0002da None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=3b4fb3599b6c4207a5d8ed34e635adbd, user.id=2def5a391d6b474cbc7027b144a818b5) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:29.234679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a28f5-15dc-4ac1-a432-f5917d0002da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:29.236109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a28f5-15dc-4ac1-a432-f5917d0002da None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:29.236413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ad4ba-8f58-4041-9ce8-ab416a799470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-1b8ad4ba-8f58-4041-9ce8-ab416a799470 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:29.241106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:29.243462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:29.244592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 545/1094] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Sep 17 03:31:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:29.251730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 550/1095] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:31:29 2022] GET /identity/v3/role_assignments?user.id=2def5a391d6b474cbc7027b144a818b5&scope.project.id=3b4fb3599b6c4207a5d8ed34e635adbd&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:29.251918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ad4ba-8f58-4041-9ce8-ab416a799470 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:29.354613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ad4ba-8f58-4041-9ce8-ab416a799470 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:29.389740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f2573-23ea-4f94-8c2a-cc6bc71d4ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:29.390368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f2573-23ea-4f94-8c2a-cc6bc71d4ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:29.390640 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f2573-23ea-4f94-8c2a-cc6bc71d4ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:29.390905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f2573-23ea-4f94-8c2a-cc6bc71d4ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:29.391569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f2573-23ea-4f94-8c2a-cc6bc71d4ee2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:29.391868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f2573-23ea-4f94-8c2a-cc6bc71d4ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:29.393247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f2573-23ea-4f94-8c2a-cc6bc71d4ee2 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:29.394870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:29.408610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 546/1096] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Sep 17 03:31:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:29.409941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a28f5-15dc-4ac1-a432-f5917d0002da None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:29.494208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:29.513356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a28f5-15dc-4ac1-a432-f5917d0002da None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:29.546771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518877f6-2ff0-4255-90af-d1a72a4afc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:29.547387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518877f6-2ff0-4255-90af-d1a72a4afc35 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:29.547661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518877f6-2ff0-4255-90af-d1a72a4afc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:29.547929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518877f6-2ff0-4255-90af-d1a72a4afc35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:29.558167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518877f6-2ff0-4255-90af-d1a72a4afc35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:29.559608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518877f6-2ff0-4255-90af-d1a72a4afc35 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-518877f6-2ff0-4255-90af-d1a72a4afc35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:29.574652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:29.575713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:29.576702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:29.576702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:29.576702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:29.576702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 551/1097] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:29.581454 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f2573-23ea-4f94-8c2a-cc6bc71d4ee2 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:29.680064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f2573-23ea-4f94-8c2a-cc6bc71d4ee2 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:29.713688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa96d9-c08e-4228-919e-3bab3c7b7df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:29.714315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa96d9-c08e-4228-919e-3bab3c7b7df7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:29.714594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa96d9-c08e-4228-919e-3bab3c7b7df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:29.714866 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa96d9-c08e-4228-919e-3bab3c7b7df7 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:29.715503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa96d9-c08e-4228-919e-3bab3c7b7df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:29.715807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa96d9-c08e-4228-919e-3bab3c7b7df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:29.717165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa96d9-c08e-4228-919e-3bab3c7b7df7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:29.728200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 547/1098] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:29.733788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518877f6-2ff0-4255-90af-d1a72a4afc35 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:29.841929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518877f6-2ff0-4255-90af-d1a72a4afc35 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:29.870162 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:29.874840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9be8c-4eed-4ac7-bfc1-ffb44f73c47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:29.875457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9be8c-4eed-4ac7-bfc1-ffb44f73c47d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:29.875734 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9be8c-4eed-4ac7-bfc1-ffb44f73c47d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:29.876006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9be8c-4eed-4ac7-bfc1-ffb44f73c47d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:29.883244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9be8c-4eed-4ac7-bfc1-ffb44f73c47d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:29.884655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9be8c-4eed-4ac7-bfc1-ffb44f73c47d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f2f9be8c-4eed-4ac7-bfc1-ffb44f73c47d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:31:29.895336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:29.896472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:29.897480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 552/1099] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Sep 17 03:31:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:29.902866 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa96d9-c08e-4228-919e-3bab3c7b7df7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:29.957869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:30.001083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa96d9-c08e-4228-919e-3bab3c7b7df7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:30.034291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd187dc1-3ab4-418f-90d6-656335d6f0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:30.034893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd187dc1-3ab4-418f-90d6-656335d6f0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:30.035168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd187dc1-3ab4-418f-90d6-656335d6f0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:30.035443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd187dc1-3ab4-418f-90d6-656335d6f0f6 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:30.036160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd187dc1-3ab4-418f-90d6-656335d6f0f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:30.036466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd187dc1-3ab4-418f-90d6-656335d6f0f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:30.037938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd187dc1-3ab4-418f-90d6-656335d6f0f6 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:30.055297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 548/1100] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Sep 17 03:31:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:30.061200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9be8c-4eed-4ac7-bfc1-ffb44f73c47d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:30.158576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9be8c-4eed-4ac7-bfc1-ffb44f73c47d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:30.192638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5ef07-3c3c-49be-9cf8-e2567a3fcde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:30.193246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5ef07-3c3c-49be-9cf8-e2567a3fcde7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:30.193526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5ef07-3c3c-49be-9cf8-e2567a3fcde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:30.193831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5ef07-3c3c-49be-9cf8-e2567a3fcde7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:30.250102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:30.259587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5ef07-3c3c-49be-9cf8-e2567a3fcde7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:30.261016 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5ef07-3c3c-49be-9cf8-e2567a3fcde7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:30.356228 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 553/1101] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:31:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:31:30.362075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd187dc1-3ab4-418f-90d6-656335d6f0f6 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:30.457408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd187dc1-3ab4-418f-90d6-656335d6f0f6 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:30.471510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:30.491179 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc2353-58db-41de-adb8-4d7afe529bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:30.492383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc2353-58db-41de-adb8-4d7afe529bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:30.492696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc2353-58db-41de-adb8-4d7afe529bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:30.492961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc2353-58db-41de-adb8-4d7afe529bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:30.502888 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc2353-58db-41de-adb8-4d7afe529bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:30.504557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc2353-58db-41de-adb8-4d7afe529bd5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-facc2353-58db-41de-adb8-4d7afe529bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:31:30.519151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:30.520167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:31:30.521365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:31:30.521365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:31:30.521365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:30.521365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 549/1102] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:30.525938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5ef07-3c3c-49be-9cf8-e2567a3fcde7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:30.613806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:30.622216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5ef07-3c3c-49be-9cf8-e2567a3fcde7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:30.655103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfce7a-2712-472d-9f8f-7c856c388325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:30.655690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfce7a-2712-472d-9f8f-7c856c388325 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:30.655968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfce7a-2712-472d-9f8f-7c856c388325 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:30.656235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfce7a-2712-472d-9f8f-7c856c388325 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:30.656857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfce7a-2712-472d-9f8f-7c856c388325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:30.657153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfce7a-2712-472d-9f8f-7c856c388325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:30.658521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfce7a-2712-472d-9f8f-7c856c388325 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:30.670469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 554/1103] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:30.675929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc2353-58db-41de-adb8-4d7afe529bd5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:30.774669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc2353-58db-41de-adb8-4d7afe529bd5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:30.811039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcc9d3-9df4-4d9f-9f78-7b2dfbbfed2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:30.811666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcc9d3-9df4-4d9f-9f78-7b2dfbbfed2c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:30.811945 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcc9d3-9df4-4d9f-9f78-7b2dfbbfed2c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:30.812217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcc9d3-9df4-4d9f-9f78-7b2dfbbfed2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:30.818834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcc9d3-9df4-4d9f-9f78-7b2dfbbfed2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:30.821126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcc9d3-9df4-4d9f-9f78-7b2dfbbfed2c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-b9dcc9d3-9df4-4d9f-9f78-7b2dfbbfed2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:31:30.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:30.833298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:30.834263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:31:30.834263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:31:30.834263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:31:30.834263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:31:30.834263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:31:30.834263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:31:30.834263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:31:30.834263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:30.834263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 550/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:31:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:31:30.839392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfce7a-2712-472d-9f8f-7c856c388325 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:30.948528 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfce7a-2712-472d-9f8f-7c856c388325 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:30.982513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb0b42-a406-4d2c-9c4e-8a77e03d389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:30.983116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0b42-a406-4d2c-9c4e-8a77e03d389a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:30.983400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0b42-a406-4d2c-9c4e-8a77e03d389a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:30.983677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0b42-a406-4d2c-9c4e-8a77e03d389a None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:30.984364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb0b42-a406-4d2c-9c4e-8a77e03d389a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:30.984672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb0b42-a406-4d2c-9c4e-8a77e03d389a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:30.986133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb0b42-a406-4d2c-9c4e-8a77e03d389a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:31.003185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 555/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:31:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:31.010415 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcc9d3-9df4-4d9f-9f78-7b2dfbbfed2c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:31.039175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:31.110490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcc9d3-9df4-4d9f-9f78-7b2dfbbfed2c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:31.143427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ced4b1-a651-4ca2-8251-9f408e2827c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:31.144014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ced4b1-a651-4ca2-8251-9f408e2827c0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:31.144286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ced4b1-a651-4ca2-8251-9f408e2827c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:31.144553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ced4b1-a651-4ca2-8251-9f408e2827c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:31.145259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ced4b1-a651-4ca2-8251-9f408e2827c0 None admin] RBAC: Adding query filter params (role.id=e88868b1e3fc44e6aee8a63eb09ddd9b, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=9901608b62344115aa561d00bfe88c1c) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:31.145560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ced4b1-a651-4ca2-8251-9f408e2827c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:31.147000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ced4b1-a651-4ca2-8251-9f408e2827c0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:31.153244 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 551/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:31:31 2022] GET /identity/v3/role_assignments?user.id=9901608b62344115aa561d00bfe88c1c&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e88868b1e3fc44e6aee8a63eb09ddd9b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:31.487140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb3d7d-96ee-435c-b433-8f39d3e127eb None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:31.487431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb3d7d-96ee-435c-b433-8f39d3e127eb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:31.487712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb3d7d-96ee-435c-b433-8f39d3e127eb None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:31.489078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 556/1107] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:31.495299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60865781-b0fc-41a7-8e33-af456c0994c0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:31.495584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60865781-b0fc-41a7-8e33-af456c0994c0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:31.495856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60865781-b0fc-41a7-8e33-af456c0994c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:31.540880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-60865781-b0fc-41a7-8e33-af456c0994c0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:31.542095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-60865781-b0fc-41a7-8e33-af456c0994c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:31.611530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 552/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:31.894429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb3d7d-96ee-435c-b433-8f39d3e127eb None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:31.910134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:31.998283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb3d7d-96ee-435c-b433-8f39d3e127eb None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:32.032370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0879a08-0fe0-4bae-a7bb-bbf4c518481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:32.032994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0879a08-0fe0-4bae-a7bb-bbf4c518481b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:32.033273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0879a08-0fe0-4bae-a7bb-bbf4c518481b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:32.033546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0879a08-0fe0-4bae-a7bb-bbf4c518481b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:32.034671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0879a08-0fe0-4bae-a7bb-bbf4c518481b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:32.036086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0879a08-0fe0-4bae-a7bb-bbf4c518481b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:32.037643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 557/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:32.044033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60865781-b0fc-41a7-8e33-af456c0994c0 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:32.064409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:32.138574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60865781-b0fc-41a7-8e33-af456c0994c0 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:32.171466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db941d0b-27bc-4e08-bfc3-7ccad35a6586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:32.172057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db941d0b-27bc-4e08-bfc3-7ccad35a6586 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:32.172332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db941d0b-27bc-4e08-bfc3-7ccad35a6586 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:32.172604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db941d0b-27bc-4e08-bfc3-7ccad35a6586 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:32.173248 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db941d0b-27bc-4e08-bfc3-7ccad35a6586 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:32.174686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db941d0b-27bc-4e08-bfc3-7ccad35a6586 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:32.250680 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 553/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:31:32 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:34.453562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92886c9e-2854-49d4-bd88-5ddd5a1e24de None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:34.454131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92886c9e-2854-49d4-bd88-5ddd5a1e24de None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:34.454131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92886c9e-2854-49d4-bd88-5ddd5a1e24de None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:34.455463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 558/1111] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:34.461168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706e7d7-d63c-48e7-8bd9-e3b3bcc8a5bb None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:34.461587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706e7d7-d63c-48e7-8bd9-e3b3bcc8a5bb None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:34.464891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706e7d7-d63c-48e7-8bd9-e3b3bcc8a5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:34.480097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-7706e7d7-d63c-48e7-8bd9-e3b3bcc8a5bb None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:34.481279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7706e7d7-d63c-48e7-8bd9-e3b3bcc8a5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:34.545629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 554/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:31:34.812638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92886c9e-2854-49d4-bd88-5ddd5a1e24de None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:34.892143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:34.910750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92886c9e-2854-49d4-bd88-5ddd5a1e24de None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:34.943801 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd155d6-1959-4711-b936-ea6826dac439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:34.944406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd155d6-1959-4711-b936-ea6826dac439 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:34.944769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd155d6-1959-4711-b936-ea6826dac439 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:34.945042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd155d6-1959-4711-b936-ea6826dac439 None admin] PATH_INFO: `/v3/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:34.946532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd155d6-1959-4711-b936-ea6826dac439 None admin] RBAC: Authorizing `identity:get_role(role_id=ec3ad2e5f7314c3b8d35826cfd81b5ba)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:34.947877 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd155d6-1959-4711-b936-ea6826dac439 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:34.949415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 559/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:34 2022] GET /identity/v3/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:34.955166 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706e7d7-d63c-48e7-8bd9-e3b3bcc8a5bb None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:34.964017 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:35.058197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706e7d7-d63c-48e7-8bd9-e3b3bcc8a5bb None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:35.092346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819142ac-a1bc-42d5-a6d4-3f4f25b3dd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:35.092966 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819142ac-a1bc-42d5-a6d4-3f4f25b3dd00 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:35.093241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819142ac-a1bc-42d5-a6d4-3f4f25b3dd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:35.093511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819142ac-a1bc-42d5-a6d4-3f4f25b3dd00 None admin] PATH_INFO: `/v3/users/41c9afe138d74b40a8f6c66d5dddb1f6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:35.106964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819142ac-a1bc-42d5-a6d4-3f4f25b3dd00 None admin] RBAC: Authorizing `identity:get_user(user_id=41c9afe138d74b40a8f6c66d5dddb1f6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:35.108392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819142ac-a1bc-42d5-a6d4-3f4f25b3dd00 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:35.110011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 555/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:34 2022] GET /identity/v3/users/41c9afe138d74b40a8f6c66d5dddb1f6 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:35.115754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd155d6-1959-4711-b936-ea6826dac439 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:35.213520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd155d6-1959-4711-b936-ea6826dac439 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:35.245319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7749f-e92c-420c-b961-1176974853f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:35.245921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7749f-e92c-420c-b961-1176974853f4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:35.246199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7749f-e92c-420c-b961-1176974853f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:35.246469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7749f-e92c-420c-b961-1176974853f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:35.263308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:35.311507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7749f-e92c-420c-b961-1176974853f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:35.312863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7749f-e92c-420c-b961-1176974853f4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:35.412114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 560/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:31:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:31:35.418656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819142ac-a1bc-42d5-a6d4-3f4f25b3dd00 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:35.478643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:35.514696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819142ac-a1bc-42d5-a6d4-3f4f25b3dd00 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:35.547146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529da4e6-940d-4708-8a6a-5919fa0bbfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:35.547747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529da4e6-940d-4708-8a6a-5919fa0bbfe0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:35.548019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529da4e6-940d-4708-8a6a-5919fa0bbfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:35.548286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529da4e6-940d-4708-8a6a-5919fa0bbfe0 None admin] PATH_INFO: `/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:35.549371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529da4e6-940d-4708-8a6a-5919fa0bbfe0 None admin] RBAC: Authorizing `identity:get_project(project_id=3b4fb3599b6c4207a5d8ed34e635adbd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:35.550831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529da4e6-940d-4708-8a6a-5919fa0bbfe0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:35.552413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 556/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:31:35 2022] GET /identity/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:35.559159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7749f-e92c-420c-b961-1176974853f4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:35.606196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:35.656544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7749f-e92c-420c-b961-1176974853f4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:35.690096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11072a-5bc9-45d7-b9d1-ed19d91b2ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:35.690683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11072a-5bc9-45d7-b9d1-ed19d91b2ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:35.690957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11072a-5bc9-45d7-b9d1-ed19d91b2ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:35.691226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11072a-5bc9-45d7-b9d1-ed19d91b2ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:35.691930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11072a-5bc9-45d7-b9d1-ed19d91b2ea0 None admin] RBAC: Adding query filter params (role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba, scope.project.id=3b4fb3599b6c4207a5d8ed34e635adbd, user.id=41c9afe138d74b40a8f6c66d5dddb1f6) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:35.692236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11072a-5bc9-45d7-b9d1-ed19d91b2ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:35.693647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11072a-5bc9-45d7-b9d1-ed19d91b2ea0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:35.699284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 561/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:31:35 2022] GET /identity/v3/role_assignments?user.id=41c9afe138d74b40a8f6c66d5dddb1f6&scope.project.id=3b4fb3599b6c4207a5d8ed34e635adbd&role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:37.838882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efdbc-e728-4982-81bd-8f7076bbd4bb None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:37.839403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efdbc-e728-4982-81bd-8f7076bbd4bb None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:37.839496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efdbc-e728-4982-81bd-8f7076bbd4bb None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:37.840768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 557/1118] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:37.845908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12d2a3-efbb-457d-8608-c1b8771e2828 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:37.846294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12d2a3-efbb-457d-8608-c1b8771e2828 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:37.846452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12d2a3-efbb-457d-8608-c1b8771e2828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:37.871392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4c12d2a3-efbb-457d-8608-c1b8771e2828 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:37.872595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4c12d2a3-efbb-457d-8608-c1b8771e2828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:37.943483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 562/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:38.209232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46efdbc-e728-4982-81bd-8f7076bbd4bb None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:38.306206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46efdbc-e728-4982-81bd-8f7076bbd4bb None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:38.333927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:38.338595 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e721a-19b4-4c55-b401-09f5d76b6ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:38.339204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e721a-19b4-4c55-b401-09f5d76b6ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:38.339481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e721a-19b4-4c55-b401-09f5d76b6ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:38.339750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e721a-19b4-4c55-b401-09f5d76b6ea8 None admin] PATH_INFO: `/v3/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:38.341124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e721a-19b4-4c55-b401-09f5d76b6ea8 None admin] RBAC: Authorizing `identity:get_role(role_id=ec3ad2e5f7314c3b8d35826cfd81b5ba)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:38.342512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e721a-19b4-4c55-b401-09f5d76b6ea8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:38.343793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 558/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:38 2022] GET /identity/v3/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:38.349557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12d2a3-efbb-457d-8608-c1b8771e2828 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:38.391526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:38.453472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12d2a3-efbb-457d-8608-c1b8771e2828 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:38.488442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e26930-173b-4fa5-a8b8-38065232f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:38.488991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26930-173b-4fa5-a8b8-38065232f302 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:38.489268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26930-173b-4fa5-a8b8-38065232f302 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:38.489537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26930-173b-4fa5-a8b8-38065232f302 None admin] PATH_INFO: `/v3/users/41c9afe138d74b40a8f6c66d5dddb1f6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:38.490661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e26930-173b-4fa5-a8b8-38065232f302 None admin] RBAC: Authorizing `identity:get_user(user_id=41c9afe138d74b40a8f6c66d5dddb1f6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:38.492909 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e26930-173b-4fa5-a8b8-38065232f302 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:38.494511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 563/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:38 2022] GET /identity/v3/users/41c9afe138d74b40a8f6c66d5dddb1f6 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:38.500355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e721a-19b4-4c55-b401-09f5d76b6ea8 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:38.603566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e721a-19b4-4c55-b401-09f5d76b6ea8 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:38.647000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ae197-33a4-4ee2-b368-fa13dfa4e01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:38.647630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ae197-33a4-4ee2-b368-fa13dfa4e01c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:38.647939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ae197-33a4-4ee2-b368-fa13dfa4e01c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:38.648209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ae197-33a4-4ee2-b368-fa13dfa4e01c None admin] PATH_INFO: `/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:38.649347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ae197-33a4-4ee2-b368-fa13dfa4e01c None admin] RBAC: Authorizing `identity:get_project(project_id=3b4fb3599b6c4207a5d8ed34e635adbd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:38.650802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ae197-33a4-4ee2-b368-fa13dfa4e01c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:38.652439 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 559/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:31:38 2022] GET /identity/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:38.658603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e26930-173b-4fa5-a8b8-38065232f302 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:38.766250 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e26930-173b-4fa5-a8b8-38065232f302 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:38.777636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:38.802590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d3bdb-e1b2-4a85-be3d-8a1bb845c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:38.803334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d3bdb-e1b2-4a85-be3d-8a1bb845c675 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:38.804038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d3bdb-e1b2-4a85-be3d-8a1bb845c675 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:38.804038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d3bdb-e1b2-4a85-be3d-8a1bb845c675 None admin] PATH_INFO: `/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd/users/41c9afe138d74b40a8f6c66d5dddb1f6/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:38.806739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d3bdb-e1b2-4a85-be3d-8a1bb845c675 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4fb3599b6c4207a5d8ed34e635adbd, user_id=41c9afe138d74b40a8f6c66d5dddb1f6, role_id=ec3ad2e5f7314c3b8d35826cfd81b5ba)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:38.808263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d3bdb-e1b2-4a85-be3d-8a1bb845c675 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:38.879659 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 564/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:31:38 2022] PUT /identity/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd/users/41c9afe138d74b40a8f6c66d5dddb1f6/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:31:41.049898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a99c8-7796-4eda-bc1e-32ad19ba8588 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:41.050440 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a99c8-7796-4eda-bc1e-32ad19ba8588 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:41.050440 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a99c8-7796-4eda-bc1e-32ad19ba8588 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:41.051685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 560/1124] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:41.056985 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f73d7-ee1d-46ef-9112-f59a988db1b6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:41.057408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f73d7-ee1d-46ef-9112-f59a988db1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:41.057515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f73d7-ee1d-46ef-9112-f59a988db1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:41.077001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ca9f73d7-ee1d-46ef-9112-f59a988db1b6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:41.078202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ca9f73d7-ee1d-46ef-9112-f59a988db1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:41.142017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 565/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:41.414029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a99c8-7796-4eda-bc1e-32ad19ba8588 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:41.479573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:41.511651 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a99c8-7796-4eda-bc1e-32ad19ba8588 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:41.544686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568bfe1-7057-4c6e-9e52-561ba11f1d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:41.545307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568bfe1-7057-4c6e-9e52-561ba11f1d13 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:41.545585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568bfe1-7057-4c6e-9e52-561ba11f1d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:41.545889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568bfe1-7057-4c6e-9e52-561ba11f1d13 None admin] PATH_INFO: `/v3/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:41.547284 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568bfe1-7057-4c6e-9e52-561ba11f1d13 None admin] RBAC: Authorizing `identity:get_role(role_id=ec3ad2e5f7314c3b8d35826cfd81b5ba)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:41.548647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568bfe1-7057-4c6e-9e52-561ba11f1d13 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:41.549983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 561/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:41 2022] GET /identity/v3/roles/ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:41.555786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f73d7-ee1d-46ef-9112-f59a988db1b6 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:41.649520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:41.652443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f73d7-ee1d-46ef-9112-f59a988db1b6 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:41.685085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a4a2c-1c20-4d8f-b3d3-6676fc23f951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:41.685635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a4a2c-1c20-4d8f-b3d3-6676fc23f951 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:41.685941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a4a2c-1c20-4d8f-b3d3-6676fc23f951 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:41.686217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a4a2c-1c20-4d8f-b3d3-6676fc23f951 None admin] PATH_INFO: `/v3/users/41c9afe138d74b40a8f6c66d5dddb1f6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:41.687319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a4a2c-1c20-4d8f-b3d3-6676fc23f951 None admin] RBAC: Authorizing `identity:get_user(user_id=41c9afe138d74b40a8f6c66d5dddb1f6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:41.692780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a4a2c-1c20-4d8f-b3d3-6676fc23f951 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:41.692780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 566/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:41 2022] GET /identity/v3/users/41c9afe138d74b40a8f6c66d5dddb1f6 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:41.696051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568bfe1-7057-4c6e-9e52-561ba11f1d13 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:41.793283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568bfe1-7057-4c6e-9e52-561ba11f1d13 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:41.826799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971b337-c69f-487a-9e16-8c0a527a51e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:41.827399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971b337-c69f-487a-9e16-8c0a527a51e7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:41.827677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971b337-c69f-487a-9e16-8c0a527a51e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:41.827949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971b337-c69f-487a-9e16-8c0a527a51e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:41.831292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:41.903214 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971b337-c69f-487a-9e16-8c0a527a51e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:41.905669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971b337-c69f-487a-9e16-8c0a527a51e7 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:41.999955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 562/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:31:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:31:42.005768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a4a2c-1c20-4d8f-b3d3-6676fc23f951 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:42.102777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a4a2c-1c20-4d8f-b3d3-6676fc23f951 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:42.135571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f0312-3c34-4b0a-9481-644a74816660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:42.136186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f0312-3c34-4b0a-9481-644a74816660 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:42.136474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f0312-3c34-4b0a-9481-644a74816660 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:42.136752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f0312-3c34-4b0a-9481-644a74816660 None admin] PATH_INFO: `/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:42.137889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f0312-3c34-4b0a-9481-644a74816660 None admin] RBAC: Authorizing `identity:get_project(project_id=3b4fb3599b6c4207a5d8ed34e635adbd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:42.139304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f0312-3c34-4b0a-9481-644a74816660 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:42.140912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 567/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:31:42 2022] GET /identity/v3/projects/3b4fb3599b6c4207a5d8ed34e635adbd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:42.147036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971b337-c69f-487a-9e16-8c0a527a51e7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:42.180481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:42.241949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971b337-c69f-487a-9e16-8c0a527a51e7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:42.274200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36e323-b37d-4fe6-bd1a-5876a64e3b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:42.274721 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36e323-b37d-4fe6-bd1a-5876a64e3b08 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:42.274995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36e323-b37d-4fe6-bd1a-5876a64e3b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:42.275260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36e323-b37d-4fe6-bd1a-5876a64e3b08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:42.275967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36e323-b37d-4fe6-bd1a-5876a64e3b08 None admin] RBAC: Adding query filter params (role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba, scope.project.id=3b4fb3599b6c4207a5d8ed34e635adbd, user.id=41c9afe138d74b40a8f6c66d5dddb1f6) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:42.276267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36e323-b37d-4fe6-bd1a-5876a64e3b08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:42.277675 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36e323-b37d-4fe6-bd1a-5876a64e3b08 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:42.284961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 563/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:31:42 2022] GET /identity/v3/role_assignments?user.id=41c9afe138d74b40a8f6c66d5dddb1f6&scope.project.id=3b4fb3599b6c4207a5d8ed34e635adbd&role.id=ec3ad2e5f7314c3b8d35826cfd81b5ba => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:44.468399 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db850b87-d4d9-46de-b72f-b20f6d9970a8 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:44.468933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db850b87-d4d9-46de-b72f-b20f6d9970a8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:44.468933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db850b87-d4d9-46de-b72f-b20f6d9970a8 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:44.470290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 568/1131] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:44.476179 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde68ca-f840-4cf9-b835-79c94dab0a56 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:44.476552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde68ca-f840-4cf9-b835-79c94dab0a56 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:44.476750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde68ca-f840-4cf9-b835-79c94dab0a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:44.495230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4bde68ca-f840-4cf9-b835-79c94dab0a56 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:44.496422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4bde68ca-f840-4cf9-b835-79c94dab0a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:44.561225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 564/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:44.833537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db850b87-d4d9-46de-b72f-b20f6d9970a8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:44.870299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:44.934711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db850b87-d4d9-46de-b72f-b20f6d9970a8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:44.968971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c3d24-327b-4928-a377-9bf7e03ecd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:44.969589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c3d24-327b-4928-a377-9bf7e03ecd4a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:44.969896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c3d24-327b-4928-a377-9bf7e03ecd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:44.970168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c3d24-327b-4928-a377-9bf7e03ecd4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:44.971259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c3d24-327b-4928-a377-9bf7e03ecd4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:44.972669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c3d24-327b-4928-a377-9bf7e03ecd4a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:44.974263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 569/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:44.980362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde68ca-f840-4cf9-b835-79c94dab0a56 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:45.011964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:45.079098 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde68ca-f840-4cf9-b835-79c94dab0a56 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:45.111789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93581e-d600-4b73-b8b3-8c9a6502c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:45.112389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93581e-d600-4b73-b8b3-8c9a6502c760 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:45.112669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93581e-d600-4b73-b8b3-8c9a6502c760 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:45.112941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93581e-d600-4b73-b8b3-8c9a6502c760 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:45.113583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93581e-d600-4b73-b8b3-8c9a6502c760 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:45.115039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93581e-d600-4b73-b8b3-8c9a6502c760 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:45.189231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 565/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:31:44 2022] POST /identity/v3/projects => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:31:47.343145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ef931-ea49-436f-9a83-9490d3fca152 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:47.343702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ef931-ea49-436f-9a83-9490d3fca152 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:47.343702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ef931-ea49-436f-9a83-9490d3fca152 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:47.345013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 570/1135] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:47.350165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab70aa-b208-4c23-9068-c37f34909c1f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:47.350536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab70aa-b208-4c23-9068-c37f34909c1f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:47.350767 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab70aa-b208-4c23-9068-c37f34909c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:47.368624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-edab70aa-b208-4c23-9068-c37f34909c1f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:47.369816 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-edab70aa-b208-4c23-9068-c37f34909c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:47.433891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 566/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:47.699336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ef931-ea49-436f-9a83-9490d3fca152 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:47.798111 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ef931-ea49-436f-9a83-9490d3fca152 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:47.800490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:47.831569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af1cf3-68f6-45f4-9642-61290c6c2ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:47.832184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af1cf3-68f6-45f4-9642-61290c6c2ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:47.832508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af1cf3-68f6-45f4-9642-61290c6c2ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:47.832983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af1cf3-68f6-45f4-9642-61290c6c2ad6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:47.834463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af1cf3-68f6-45f4-9642-61290c6c2ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:47.836490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af1cf3-68f6-45f4-9642-61290c6c2ad6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:47.838063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 571/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:31:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:47.843760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab70aa-b208-4c23-9068-c37f34909c1f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:47.857951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:47.941024 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab70aa-b208-4c23-9068-c37f34909c1f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:47.974681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54bbec-3238-4aa6-a3c4-a601b7585d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:47.975205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54bbec-3238-4aa6-a3c4-a601b7585d46 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:47.975478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54bbec-3238-4aa6-a3c4-a601b7585d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:47.976391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54bbec-3238-4aa6-a3c4-a601b7585d46 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:47.977073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54bbec-3238-4aa6-a3c4-a601b7585d46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:47.978529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54bbec-3238-4aa6-a3c4-a601b7585d46 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:48.055707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 567/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:31:47 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:31:50.274141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712d8bf-f6b9-4b41-8d9b-bd2d7f91b609 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:50.274670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712d8bf-f6b9-4b41-8d9b-bd2d7f91b609 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:50.274670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712d8bf-f6b9-4b41-8d9b-bd2d7f91b609 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:50.276017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 572/1139] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:50.281168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79094f-34fa-4d65-8d9a-1be309797c52 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:50.281537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79094f-34fa-4d65-8d9a-1be309797c52 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:50.281708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79094f-34fa-4d65-8d9a-1be309797c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:50.299523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-8e79094f-34fa-4d65-8d9a-1be309797c52 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:50.300688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8e79094f-34fa-4d65-8d9a-1be309797c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:50.361915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:50.368521 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 568/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:50.644931 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712d8bf-f6b9-4b41-8d9b-bd2d7f91b609 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:50.764068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712d8bf-f6b9-4b41-8d9b-bd2d7f91b609 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:50.795797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50631ef-216e-41cb-a442-276d5be0d2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:50.796432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50631ef-216e-41cb-a442-276d5be0d2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:50.796710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50631ef-216e-41cb-a442-276d5be0d2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:50.796978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50631ef-216e-41cb-a442-276d5be0d2c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:50.807548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50631ef-216e-41cb-a442-276d5be0d2c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e50631ef-216e-41cb-a442-276d5be0d2c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:50.812235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:50.813327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:50.814457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 573/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:31:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:50.820108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79094f-34fa-4d65-8d9a-1be309797c52 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:50.914372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79094f-34fa-4d65-8d9a-1be309797c52 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:50.946997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86895ae8-8dd1-41fa-9a72-c5bb7bca5c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:50.947586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86895ae8-8dd1-41fa-9a72-c5bb7bca5c04 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:50.947861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86895ae8-8dd1-41fa-9a72-c5bb7bca5c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:50.948137 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86895ae8-8dd1-41fa-9a72-c5bb7bca5c04 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:50.949045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86895ae8-8dd1-41fa-9a72-c5bb7bca5c04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:50.949344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86895ae8-8dd1-41fa-9a72-c5bb7bca5c04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:50.950735 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86895ae8-8dd1-41fa-9a72-c5bb7bca5c04 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:50.958476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 569/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:50.964797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50631ef-216e-41cb-a442-276d5be0d2c8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:50.996160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:51.064405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50631ef-216e-41cb-a442-276d5be0d2c8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:51.097411 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2da98-e6cb-4e2b-8010-cb84673d3e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:51.098048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2da98-e6cb-4e2b-8010-cb84673d3e46 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:51.098327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2da98-e6cb-4e2b-8010-cb84673d3e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:51.098596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2da98-e6cb-4e2b-8010-cb84673d3e46 None admin] PATH_INFO: `/v3/users/984ea36ea36242e8bfbc4812b22d292d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:51.111266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2da98-e6cb-4e2b-8010-cb84673d3e46 None admin] RBAC: Authorizing `identity:get_user(user_id=984ea36ea36242e8bfbc4812b22d292d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:51.112674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2da98-e6cb-4e2b-8010-cb84673d3e46 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:51.114274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 574/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:50 2022] GET /identity/v3/users/984ea36ea36242e8bfbc4812b22d292d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:51.120142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86895ae8-8dd1-41fa-9a72-c5bb7bca5c04 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:51.178427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:51.218310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86895ae8-8dd1-41fa-9a72-c5bb7bca5c04 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:51.251451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758cc27-73c1-4240-b838-42fcf9021a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:51.252046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758cc27-73c1-4240-b838-42fcf9021a4e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:51.252324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758cc27-73c1-4240-b838-42fcf9021a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:51.252600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758cc27-73c1-4240-b838-42fcf9021a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:51.317874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758cc27-73c1-4240-b838-42fcf9021a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:51.319453 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758cc27-73c1-4240-b838-42fcf9021a4e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:51.374359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:51.413882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 570/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:31:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:31:51.419796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2da98-e6cb-4e2b-8010-cb84673d3e46 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:51.508713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:51.517711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2da98-e6cb-4e2b-8010-cb84673d3e46 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:51.550842 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df134e8c-3e89-4e05-8485-c9a6a9ee5ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:51.551444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df134e8c-3e89-4e05-8485-c9a6a9ee5ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:51.551722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df134e8c-3e89-4e05-8485-c9a6a9ee5ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:51.551995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df134e8c-3e89-4e05-8485-c9a6a9ee5ed1 None admin] PATH_INFO: `/v3/projects/3294340b249f4349bf54f8052b7bb7e4` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:51.553077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df134e8c-3e89-4e05-8485-c9a6a9ee5ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=3294340b249f4349bf54f8052b7bb7e4)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:51.554630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df134e8c-3e89-4e05-8485-c9a6a9ee5ed1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:51.556326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 575/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:31:51 2022] GET /identity/v3/projects/3294340b249f4349bf54f8052b7bb7e4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:51.562416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758cc27-73c1-4240-b838-42fcf9021a4e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:51.658056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758cc27-73c1-4240-b838-42fcf9021a4e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:51.690559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f75c1e-506e-4f86-a6a6-28e37726b2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:51.691149 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f75c1e-506e-4f86-a6a6-28e37726b2fe None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:51.691428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f75c1e-506e-4f86-a6a6-28e37726b2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:51.691699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f75c1e-506e-4f86-a6a6-28e37726b2fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:51.692411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f75c1e-506e-4f86-a6a6-28e37726b2fe None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=3294340b249f4349bf54f8052b7bb7e4, user.id=984ea36ea36242e8bfbc4812b22d292d) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:51.692717 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f75c1e-506e-4f86-a6a6-28e37726b2fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:51.694197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f75c1e-506e-4f86-a6a6-28e37726b2fe None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:51.699603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 571/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:31:51 2022] GET /identity/v3/role_assignments?user.id=984ea36ea36242e8bfbc4812b22d292d&scope.project.id=3294340b249f4349bf54f8052b7bb7e4&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:31:53.801125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fdbe4-be53-41dc-a929-b3e67780f5c5 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:53.801493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fdbe4-be53-41dc-a929-b3e67780f5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:53.801614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fdbe4-be53-41dc-a929-b3e67780f5c5 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:53.802965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 576/1147] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:53.808904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c5b33-773c-42e3-8e64-853fb92fa28f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:53.809335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c5b33-773c-42e3-8e64-853fb92fa28f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:53.809429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c5b33-773c-42e3-8e64-853fb92fa28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:53.828763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-951c5b33-773c-42e3-8e64-853fb92fa28f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:53.829950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-951c5b33-773c-42e3-8e64-853fb92fa28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:53.846581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:53.893287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 572/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:54.170395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fdbe4-be53-41dc-a929-b3e67780f5c5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:54.268558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fdbe4-be53-41dc-a929-b3e67780f5c5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:54.301065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68680b95-f36f-4cfd-8a8f-2b7c69ff3481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:54.301677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68680b95-f36f-4cfd-8a8f-2b7c69ff3481 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:54.301983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68680b95-f36f-4cfd-8a8f-2b7c69ff3481 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:54.302264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68680b95-f36f-4cfd-8a8f-2b7c69ff3481 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:54.312061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68680b95-f36f-4cfd-8a8f-2b7c69ff3481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-68680b95-f36f-4cfd-8a8f-2b7c69ff3481 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:54.316786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:54.317907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:31:54.318993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 577/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:54.324728 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c5b33-773c-42e3-8e64-853fb92fa28f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:54.418933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c5b33-773c-42e3-8e64-853fb92fa28f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:54.452284 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccedf7-3f3a-4e7d-a893-91d29b25afd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:54.452825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccedf7-3f3a-4e7d-a893-91d29b25afd6 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:54.453104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccedf7-3f3a-4e7d-a893-91d29b25afd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:54.453369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccedf7-3f3a-4e7d-a893-91d29b25afd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:54.454055 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccedf7-3f3a-4e7d-a893-91d29b25afd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:54.454355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccedf7-3f3a-4e7d-a893-91d29b25afd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:54.455711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccedf7-3f3a-4e7d-a893-91d29b25afd6 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:54.464677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 573/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:54.470300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68680b95-f36f-4cfd-8a8f-2b7c69ff3481 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:54.490530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:54.572776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68680b95-f36f-4cfd-8a8f-2b7c69ff3481 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:54.607028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef412f-f60f-4d82-8719-b308d86c0155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:54.607640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef412f-f60f-4d82-8719-b308d86c0155 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:54.607919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef412f-f60f-4d82-8719-b308d86c0155 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:54.608192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef412f-f60f-4d82-8719-b308d86c0155 None admin] PATH_INFO: `/v3/users/984ea36ea36242e8bfbc4812b22d292d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:54.609298 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef412f-f60f-4d82-8719-b308d86c0155 None admin] RBAC: Authorizing `identity:get_user(user_id=984ea36ea36242e8bfbc4812b22d292d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:54.610742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef412f-f60f-4d82-8719-b308d86c0155 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:54.612300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 578/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:54 2022] GET /identity/v3/users/984ea36ea36242e8bfbc4812b22d292d => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:54.618593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccedf7-3f3a-4e7d-a893-91d29b25afd6 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:54.637018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:54.715255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccedf7-3f3a-4e7d-a893-91d29b25afd6 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:54.746823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f090f-8c8d-41f0-8727-8a8bcf1c1271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:54.747419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f090f-8c8d-41f0-8727-8a8bcf1c1271 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:54.747692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f090f-8c8d-41f0-8727-8a8bcf1c1271 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:54.747959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f090f-8c8d-41f0-8727-8a8bcf1c1271 None admin] PATH_INFO: `/v3/projects/3294340b249f4349bf54f8052b7bb7e4` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:54.749044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f090f-8c8d-41f0-8727-8a8bcf1c1271 None admin] RBAC: Authorizing `identity:get_project(project_id=3294340b249f4349bf54f8052b7bb7e4)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:54.750488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f090f-8c8d-41f0-8727-8a8bcf1c1271 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:54.752063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 574/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:31:54 2022] GET /identity/v3/projects/3294340b249f4349bf54f8052b7bb7e4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:54.758579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef412f-f60f-4d82-8719-b308d86c0155 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:54.840662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:54.855484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef412f-f60f-4d82-8719-b308d86c0155 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:54.887770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ac250-5b2f-4063-ac89-1641011a380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:54.888414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ac250-5b2f-4063-ac89-1641011a380c None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:54.888691 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ac250-5b2f-4063-ac89-1641011a380c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:54.888965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ac250-5b2f-4063-ac89-1641011a380c None admin] PATH_INFO: `/v3/projects/3294340b249f4349bf54f8052b7bb7e4/users/984ea36ea36242e8bfbc4812b22d292d/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:54.890836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ac250-5b2f-4063-ac89-1641011a380c None admin] RBAC: Authorizing `identity:create_grant(project_id=3294340b249f4349bf54f8052b7bb7e4, user_id=984ea36ea36242e8bfbc4812b22d292d, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:54.892235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ac250-5b2f-4063-ac89-1641011a380c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:54.964310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 579/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:31:54 2022] PUT /identity/v3/projects/3294340b249f4349bf54f8052b7bb7e4/users/984ea36ea36242e8bfbc4812b22d292d/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:31:57.092691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d833-5c02-4ee4-a7fd-39549e08bbe9 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:57.093237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d833-5c02-4ee4-a7fd-39549e08bbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:57.093237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d833-5c02-4ee4-a7fd-39549e08bbe9 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:57.094500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 575/1154] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:31:57.099787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5de6b-ee85-493d-bb59-f5f1943b19bd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:57.100239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5de6b-ee85-493d-bb59-f5f1943b19bd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:57.101304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5de6b-ee85-493d-bb59-f5f1943b19bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:57.120586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-63a5de6b-ee85-493d-bb59-f5f1943b19bd None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:31:57.121796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-63a5de6b-ee85-493d-bb59-f5f1943b19bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:31:57.188736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 580/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:31:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:31:57.466036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827d833-5c02-4ee4-a7fd-39549e08bbe9 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:57.548611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:57.564445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827d833-5c02-4ee4-a7fd-39549e08bbe9 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:57.597223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77715a2-54d1-4d21-804a-9d5060a48bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:57.597935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77715a2-54d1-4d21-804a-9d5060a48bda None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:57.602980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77715a2-54d1-4d21-804a-9d5060a48bda None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:57.602980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77715a2-54d1-4d21-804a-9d5060a48bda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:57.609428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77715a2-54d1-4d21-804a-9d5060a48bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-c77715a2-54d1-4d21-804a-9d5060a48bda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:31:57.614138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:31:57.615351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:31:57.616351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 576/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:31:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:31:57.621231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5de6b-ee85-493d-bb59-f5f1943b19bd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:57.694948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:57.717379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5de6b-ee85-493d-bb59-f5f1943b19bd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:57.750455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaa470-a476-4f23-a167-8e8ad6ca9ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:57.751049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa470-a476-4f23-a167-8e8ad6ca9ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:57.751329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa470-a476-4f23-a167-8e8ad6ca9ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:57.751601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa470-a476-4f23-a167-8e8ad6ca9ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:57.752253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaa470-a476-4f23-a167-8e8ad6ca9ba2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:57.752553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaa470-a476-4f23-a167-8e8ad6ca9ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:57.753927 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaa470-a476-4f23-a167-8e8ad6ca9ba2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:57.762120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 581/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:31:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:57.768006 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77715a2-54d1-4d21-804a-9d5060a48bda None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:57.865422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77715a2-54d1-4d21-804a-9d5060a48bda None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:57.898200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cce55a-8d84-4cf0-b4bc-4e6db936e76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:57.898809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cce55a-8d84-4cf0-b4bc-4e6db936e76f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:57.899087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cce55a-8d84-4cf0-b4bc-4e6db936e76f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:57.899358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cce55a-8d84-4cf0-b4bc-4e6db936e76f None admin] PATH_INFO: `/v3/users/984ea36ea36242e8bfbc4812b22d292d` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:57.900473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cce55a-8d84-4cf0-b4bc-4e6db936e76f None admin] RBAC: Authorizing `identity:get_user(user_id=984ea36ea36242e8bfbc4812b22d292d)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:57.901926 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cce55a-8d84-4cf0-b4bc-4e6db936e76f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:57.903577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 577/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:31:57 2022] GET /identity/v3/users/984ea36ea36242e8bfbc4812b22d292d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:57.909299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaa470-a476-4f23-a167-8e8ad6ca9ba2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:58.012383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaa470-a476-4f23-a167-8e8ad6ca9ba2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:58.044429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44cfa5-29f5-4cd0-9dc9-daecc6b0def4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:58.045023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44cfa5-29f5-4cd0-9dc9-daecc6b0def4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:58.045303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44cfa5-29f5-4cd0-9dc9-daecc6b0def4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:58.045577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44cfa5-29f5-4cd0-9dc9-daecc6b0def4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:58.053912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:58.110037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44cfa5-29f5-4cd0-9dc9-daecc6b0def4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:58.111468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44cfa5-29f5-4cd0-9dc9-daecc6b0def4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:58.207269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 582/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:31:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:31:58.213353 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cce55a-8d84-4cf0-b4bc-4e6db936e76f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:58.228283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:58.310795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cce55a-8d84-4cf0-b4bc-4e6db936e76f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:58.342480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117db9d-532c-4790-ac7f-bf717c1646e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:58.343083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117db9d-532c-4790-ac7f-bf717c1646e6 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:58.343355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117db9d-532c-4790-ac7f-bf717c1646e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:58.343619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117db9d-532c-4790-ac7f-bf717c1646e6 None admin] PATH_INFO: `/v3/projects/3294340b249f4349bf54f8052b7bb7e4` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:58.344702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117db9d-532c-4790-ac7f-bf717c1646e6 None admin] RBAC: Authorizing `identity:get_project(project_id=3294340b249f4349bf54f8052b7bb7e4)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:58.346144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117db9d-532c-4790-ac7f-bf717c1646e6 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:58.347720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 578/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:31:58 2022] GET /identity/v3/projects/3294340b249f4349bf54f8052b7bb7e4 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:31:58.353775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44cfa5-29f5-4cd0-9dc9-daecc6b0def4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:31:58.392300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:31:58.453710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44cfa5-29f5-4cd0-9dc9-daecc6b0def4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:31:58.485897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f35f06-1341-4a44-8e48-86fbe649ca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:31:58.486495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f35f06-1341-4a44-8e48-86fbe649ca4c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:31:58.486779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f35f06-1341-4a44-8e48-86fbe649ca4c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:31:58.487055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f35f06-1341-4a44-8e48-86fbe649ca4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:31:58.487773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f35f06-1341-4a44-8e48-86fbe649ca4c None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=3294340b249f4349bf54f8052b7bb7e4, user.id=984ea36ea36242e8bfbc4812b22d292d) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:31:58.488083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f35f06-1341-4a44-8e48-86fbe649ca4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:31:58.489492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f35f06-1341-4a44-8e48-86fbe649ca4c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:31:58.495775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 583/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:31:58 2022] GET /identity/v3/role_assignments?user.id=984ea36ea36242e8bfbc4812b22d292d&scope.project.id=3294340b249f4349bf54f8052b7bb7e4&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:00.657052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5db2f-991e-4ebc-9148-9487b93b59cf None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:00.657608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5db2f-991e-4ebc-9148-9487b93b59cf None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:00.657608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5db2f-991e-4ebc-9148-9487b93b59cf None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:00.658975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 579/1162] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:32:00.664723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd492c-1870-4a12-9fc4-2569828f808a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:00.665144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd492c-1870-4a12-9fc4-2569828f808a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:00.665271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd492c-1870-4a12-9fc4-2569828f808a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:00.683683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-edbd492c-1870-4a12-9fc4-2569828f808a None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:32:00.684855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-edbd492c-1870-4a12-9fc4-2569828f808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:32:00.750366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 584/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:32:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:32:01.016368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5db2f-991e-4ebc-9148-9487b93b59cf None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:01.094761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:01.114258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5db2f-991e-4ebc-9148-9487b93b59cf None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:01.147890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7ad8d-5c71-4648-9887-b190c6c00c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:01.148472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ad8d-5c71-4648-9887-b190c6c00c36 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:01.148745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ad8d-5c71-4648-9887-b190c6c00c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:01.149010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ad8d-5c71-4648-9887-b190c6c00c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:01.213462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7ad8d-5c71-4648-9887-b190c6c00c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:01.214916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7ad8d-5c71-4648-9887-b190c6c00c36 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:01.289306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:01.308001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 580/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:32:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:32:01.314066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd492c-1870-4a12-9fc4-2569828f808a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:01.343869 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:01.409042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd492c-1870-4a12-9fc4-2569828f808a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:01.447089 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de9301-4f4e-407d-80ed-1d7fc88e40a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:01.447717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de9301-4f4e-407d-80ed-1d7fc88e40a2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:01.447999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de9301-4f4e-407d-80ed-1d7fc88e40a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:01.448271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de9301-4f4e-407d-80ed-1d7fc88e40a2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:01.455215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de9301-4f4e-407d-80ed-1d7fc88e40a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:01.456646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de9301-4f4e-407d-80ed-1d7fc88e40a2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-64de9301-4f4e-407d-80ed-1d7fc88e40a2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:32:01.467878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:32:01.469103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:32:01.470180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 585/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Sep 17 03:32:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:32:01.475214 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7ad8d-5c71-4648-9887-b190c6c00c36 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:01.571110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7ad8d-5c71-4648-9887-b190c6c00c36 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:01.604455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11cc63-9242-48b8-a218-998dcaf36b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:01.605057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11cc63-9242-48b8-a218-998dcaf36b71 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:01.605335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11cc63-9242-48b8-a218-998dcaf36b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:01.605632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11cc63-9242-48b8-a218-998dcaf36b71 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:01.606299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11cc63-9242-48b8-a218-998dcaf36b71 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:32:01.606602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11cc63-9242-48b8-a218-998dcaf36b71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:01.607961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11cc63-9242-48b8-a218-998dcaf36b71 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:01.614954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 581/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Sep 17 03:32:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:03.779051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732fa79-be1b-4e90-9a20-ebf056041613 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:03.779615 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732fa79-be1b-4e90-9a20-ebf056041613 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:03.779615 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732fa79-be1b-4e90-9a20-ebf056041613 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:03.780936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 586/1167] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:32:03.786290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b19faa-089c-43f2-abdb-0b1ddc93e1f0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:03.786723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b19faa-089c-43f2-abdb-0b1ddc93e1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:03.786823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b19faa-089c-43f2-abdb-0b1ddc93e1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:03.805396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-62b19faa-089c-43f2-abdb-0b1ddc93e1f0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:32:03.806602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-62b19faa-089c-43f2-abdb-0b1ddc93e1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:32:03.845041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:03.870120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 582/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:32:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:32:04.150227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732fa79-be1b-4e90-9a20-ebf056041613 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:04.239482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:04.249217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732fa79-be1b-4e90-9a20-ebf056041613 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:04.282747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc158515-f2cf-4d59-a915-1e6fc6622159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:04.283344 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc158515-f2cf-4d59-a915-1e6fc6622159 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:04.283628 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc158515-f2cf-4d59-a915-1e6fc6622159 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:04.283907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc158515-f2cf-4d59-a915-1e6fc6622159 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:04.284578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc158515-f2cf-4d59-a915-1e6fc6622159 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:04.285960 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc158515-f2cf-4d59-a915-1e6fc6622159 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:04.355263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 587/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Sat Sep 17 03:32:04 2022] POST /identity/v3/domains => generated 267 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:32:06.522581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5953cd5-30e0-4cf3-a30e-2bf2ed352c63 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:06.524121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5953cd5-30e0-4cf3-a30e-2bf2ed352c63 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:06.524121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5953cd5-30e0-4cf3-a30e-2bf2ed352c63 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:06.525092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 583/1170] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:32:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:32:06.530393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65773323-c35a-4a6f-8d3a-7e0d063d9277 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:06.530783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65773323-c35a-4a6f-8d3a-7e0d063d9277 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:06.530948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65773323-c35a-4a6f-8d3a-7e0d063d9277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:06.549174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-65773323-c35a-4a6f-8d3a-7e0d063d9277 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:32:06.550379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-65773323-c35a-4a6f-8d3a-7e0d063d9277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:32:06.614062 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 588/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:32:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:32:06.890129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5953cd5-30e0-4cf3-a30e-2bf2ed352c63 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:06.987605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5953cd5-30e0-4cf3-a30e-2bf2ed352c63 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:07.020405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff87c2-357e-4253-b317-c2f797e8f81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:07.021010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff87c2-357e-4253-b317-c2f797e8f81e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:07.021283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff87c2-357e-4253-b317-c2f797e8f81e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:07.021549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff87c2-357e-4253-b317-c2f797e8f81e None admin] PATH_INFO: `/v3/domains/10730519002f49cd94f57cb5942a0266` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:07.032091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff87c2-357e-4253-b317-c2f797e8f81e None admin] RBAC: Authorizing `identity:get_domain(domain_id=10730519002f49cd94f57cb5942a0266)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:07.033517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff87c2-357e-4253-b317-c2f797e8f81e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:07.035083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 584/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Sat Sep 17 03:32:06 2022] GET /identity/v3/domains/10730519002f49cd94f57cb5942a0266 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:07.041260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65773323-c35a-4a6f-8d3a-7e0d063d9277 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:07.120172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:07.137858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65773323-c35a-4a6f-8d3a-7e0d063d9277 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:07.171170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007f60d-43bc-4772-a979-2cb558b900f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:07.171769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007f60d-43bc-4772-a979-2cb558b900f4 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:07.172053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007f60d-43bc-4772-a979-2cb558b900f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:07.172329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007f60d-43bc-4772-a979-2cb558b900f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:07.172986 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007f60d-43bc-4772-a979-2cb558b900f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:07.174483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007f60d-43bc-4772-a979-2cb558b900f4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:07.245947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 589/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:32:07 2022] POST /identity/v3/projects => generated 361 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:32:09.409471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1c4f0-7176-4f75-8d96-9807ae8086f6 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:09.410039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1c4f0-7176-4f75-8d96-9807ae8086f6 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:09.410039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1c4f0-7176-4f75-8d96-9807ae8086f6 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:09.411390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 585/1174] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:32:09.417237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e56316-dbc1-4962-b13c-06a391979fd2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:09.417652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e56316-dbc1-4962-b13c-06a391979fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:09.417813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e56316-dbc1-4962-b13c-06a391979fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:09.436524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d5e56316-dbc1-4962-b13c-06a391979fd2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:32:09.437714 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d5e56316-dbc1-4962-b13c-06a391979fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:32:09.502442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 590/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:32:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:32:09.782808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1c4f0-7176-4f75-8d96-9807ae8086f6 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:09.824324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:09.898092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1c4f0-7176-4f75-8d96-9807ae8086f6 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:09.931285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c40bec-af58-4186-bc14-e55343c71ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:09.931904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c40bec-af58-4186-bc14-e55343c71ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:09.932184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c40bec-af58-4186-bc14-e55343c71ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:09.932454 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c40bec-af58-4186-bc14-e55343c71ce0 None admin] PATH_INFO: `/v3/domains/10730519002f49cd94f57cb5942a0266` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:09.933572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c40bec-af58-4186-bc14-e55343c71ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=10730519002f49cd94f57cb5942a0266)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:09.935030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c40bec-af58-4186-bc14-e55343c71ce0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:09.936598 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 586/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Sat Sep 17 03:32:09 2022] GET /identity/v3/domains/10730519002f49cd94f57cb5942a0266 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:09.944092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e56316-dbc1-4962-b13c-06a391979fd2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:10.006716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:10.044818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e56316-dbc1-4962-b13c-06a391979fd2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:10.080369 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333f5a0-29c6-4a1b-88ae-9449306a0d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:10.080966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333f5a0-29c6-4a1b-88ae-9449306a0d47 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:10.081247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333f5a0-29c6-4a1b-88ae-9449306a0d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:10.081518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333f5a0-29c6-4a1b-88ae-9449306a0d47 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:10.082327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333f5a0-29c6-4a1b-88ae-9449306a0d47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:10.083753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333f5a0-29c6-4a1b-88ae-9449306a0d47 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:10.158952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 591/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:32:09 2022] POST /identity/v3/users => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:32:12.344323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d861ab-3ee8-44ac-bfd9-ff522c596459 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:12.344855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d861ab-3ee8-44ac-bfd9-ff522c596459 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:12.344855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d861ab-3ee8-44ac-bfd9-ff522c596459 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:12.346227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 587/1178] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:32:12.352276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb61a7-0da1-45c4-8a50-b10e0e435159 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:12.352699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb61a7-0da1-45c4-8a50-b10e0e435159 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:12.352831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb61a7-0da1-45c4-8a50-b10e0e435159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:12.371118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b8bb61a7-0da1-45c4-8a50-b10e0e435159 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:32:12.372288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b8bb61a7-0da1-45c4-8a50-b10e0e435159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:32:12.435696 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 592/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:32:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:32:12.704312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d861ab-3ee8-44ac-bfd9-ff522c596459 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:12.805113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d861ab-3ee8-44ac-bfd9-ff522c596459 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:12.806679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:12.839469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba288f4-d0ce-4a47-9582-73ce562ba2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:12.840087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba288f4-d0ce-4a47-9582-73ce562ba2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:12.840364 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba288f4-d0ce-4a47-9582-73ce562ba2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:12.840635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba288f4-d0ce-4a47-9582-73ce562ba2a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:12.850501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba288f4-d0ce-4a47-9582-73ce562ba2a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-6ba288f4-d0ce-4a47-9582-73ce562ba2a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:32:12.855769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:32:12.856846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:32:12.858072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 588/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:32:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:32:12.862904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb61a7-0da1-45c4-8a50-b10e0e435159 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:12.910936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:12.958426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb61a7-0da1-45c4-8a50-b10e0e435159 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:12.991324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207600a7-e35b-4a0e-9520-848c7d1668a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:12.991918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207600a7-e35b-4a0e-9520-848c7d1668a9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:12.992199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207600a7-e35b-4a0e-9520-848c7d1668a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:12.992473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207600a7-e35b-4a0e-9520-848c7d1668a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:12.993130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207600a7-e35b-4a0e-9520-848c7d1668a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:32:12.993432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207600a7-e35b-4a0e-9520-848c7d1668a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:12.994818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207600a7-e35b-4a0e-9520-848c7d1668a9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:13.002510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 593/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:32:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:13.008971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba288f4-d0ce-4a47-9582-73ce562ba2a7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:13.104763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba288f4-d0ce-4a47-9582-73ce562ba2a7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:13.136896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a97e1-8a3d-40b8-b3db-1e8ae245dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:13.137497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a97e1-8a3d-40b8-b3db-1e8ae245dfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:13.137794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a97e1-8a3d-40b8-b3db-1e8ae245dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:13.138063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a97e1-8a3d-40b8-b3db-1e8ae245dfb9 None admin] PATH_INFO: `/v3/users/3bac8392931845efb0b44232806b2944` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:13.150592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a97e1-8a3d-40b8-b3db-1e8ae245dfb9 None admin] RBAC: Authorizing `identity:get_user(user_id=3bac8392931845efb0b44232806b2944)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:13.152007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a97e1-8a3d-40b8-b3db-1e8ae245dfb9 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:13.153559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 589/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:32:13 2022] GET /identity/v3/users/3bac8392931845efb0b44232806b2944 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:13.159304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207600a7-e35b-4a0e-9520-848c7d1668a9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:13.254819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207600a7-e35b-4a0e-9520-848c7d1668a9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:13.266932 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:13.287139 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c86d6-4865-46df-8161-83f640bfcec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:13.287732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c86d6-4865-46df-8161-83f640bfcec1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:13.288012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c86d6-4865-46df-8161-83f640bfcec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:13.288518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c86d6-4865-46df-8161-83f640bfcec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:13.355772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c86d6-4865-46df-8161-83f640bfcec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:13.357211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c86d6-4865-46df-8161-83f640bfcec1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:13.452812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 594/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:32:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:32:13.458771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a97e1-8a3d-40b8-b3db-1e8ae245dfb9 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:13.480797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:13.555331 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a97e1-8a3d-40b8-b3db-1e8ae245dfb9 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:13.588369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59754be-ef12-432f-a447-4f5d2700cfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:13.588974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59754be-ef12-432f-a447-4f5d2700cfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:13.589247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59754be-ef12-432f-a447-4f5d2700cfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:13.589514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59754be-ef12-432f-a447-4f5d2700cfd8 None admin] PATH_INFO: `/v3/projects/96d7d51602334751ad658ced10495775` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:13.590631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59754be-ef12-432f-a447-4f5d2700cfd8 None admin] RBAC: Authorizing `identity:get_project(project_id=96d7d51602334751ad658ced10495775)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:13.592072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59754be-ef12-432f-a447-4f5d2700cfd8 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:13.593651 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 590/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:32:13 2022] GET /identity/v3/projects/96d7d51602334751ad658ced10495775 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:13.599706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c86d6-4865-46df-8161-83f640bfcec1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:13.613356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:13.695006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c86d6-4865-46df-8161-83f640bfcec1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:13.727600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7688c6b-5000-48db-9cc3-136863b18a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:13.728200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688c6b-5000-48db-9cc3-136863b18a89 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:13.728481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688c6b-5000-48db-9cc3-136863b18a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:13.728754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688c6b-5000-48db-9cc3-136863b18a89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:13.729472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7688c6b-5000-48db-9cc3-136863b18a89 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=96d7d51602334751ad658ced10495775, user.id=3bac8392931845efb0b44232806b2944) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:32:13.729806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7688c6b-5000-48db-9cc3-136863b18a89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:13.731311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7688c6b-5000-48db-9cc3-136863b18a89 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:13.736560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 595/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:32:13 2022] GET /identity/v3/role_assignments?user.id=3bac8392931845efb0b44232806b2944&scope.project.id=96d7d51602334751ad658ced10495775&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:15.865860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb0357-d2ee-474d-8ed1-eaadf59d47ad None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:15.866442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb0357-d2ee-474d-8ed1-eaadf59d47ad None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:15.866612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb0357-d2ee-474d-8ed1-eaadf59d47ad None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:15.867962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 591/1186] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:32:15.873029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dcd3d-ee5c-4e47-b89a-3ba3e374df92 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:15.873436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dcd3d-ee5c-4e47-b89a-3ba3e374df92 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:15.873595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dcd3d-ee5c-4e47-b89a-3ba3e374df92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:15.892050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e40dcd3d-ee5c-4e47-b89a-3ba3e374df92 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:32:15.893229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e40dcd3d-ee5c-4e47-b89a-3ba3e374df92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:32:15.957976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 596/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:32:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:32:16.233985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb0357-d2ee-474d-8ed1-eaadf59d47ad None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:16.318806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:16.331387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb0357-d2ee-474d-8ed1-eaadf59d47ad None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:16.364715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43af911-7c5c-4089-98c5-039bde0034a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:16.365319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43af911-7c5c-4089-98c5-039bde0034a4 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:16.365591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43af911-7c5c-4089-98c5-039bde0034a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:16.365883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43af911-7c5c-4089-98c5-039bde0034a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:16.376044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43af911-7c5c-4089-98c5-039bde0034a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-c43af911-7c5c-4089-98c5-039bde0034a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:32:16.380668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:32:16.382582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:32:16.383998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 592/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:32:16.388312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dcd3d-ee5c-4e47-b89a-3ba3e374df92 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:16.396379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:16.484915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dcd3d-ee5c-4e47-b89a-3ba3e374df92 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:16.518046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e41582-181c-4c8e-bd03-44e277fd3270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:16.518644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e41582-181c-4c8e-bd03-44e277fd3270 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:16.518923 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e41582-181c-4c8e-bd03-44e277fd3270 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:16.519196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e41582-181c-4c8e-bd03-44e277fd3270 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:16.519858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e41582-181c-4c8e-bd03-44e277fd3270 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:32:16.520159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e41582-181c-4c8e-bd03-44e277fd3270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:16.521510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e41582-181c-4c8e-bd03-44e277fd3270 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:16.529544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 597/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:16.535390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43af911-7c5c-4089-98c5-039bde0034a4 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:16.631178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43af911-7c5c-4089-98c5-039bde0034a4 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:16.663516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c7b78-435f-4107-9cda-3da1fac2eb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:16.664130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c7b78-435f-4107-9cda-3da1fac2eb53 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:16.664397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c7b78-435f-4107-9cda-3da1fac2eb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:16.664663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c7b78-435f-4107-9cda-3da1fac2eb53 None admin] PATH_INFO: `/v3/users/3bac8392931845efb0b44232806b2944` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:16.665795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c7b78-435f-4107-9cda-3da1fac2eb53 None admin] RBAC: Authorizing `identity:get_user(user_id=3bac8392931845efb0b44232806b2944)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:16.667216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c7b78-435f-4107-9cda-3da1fac2eb53 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:16.668758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 593/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:32:16 2022] GET /identity/v3/users/3bac8392931845efb0b44232806b2944 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:16.674612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e41582-181c-4c8e-bd03-44e277fd3270 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:16.739148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:16.770853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e41582-181c-4c8e-bd03-44e277fd3270 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:16.804517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409db2d-1e1d-4522-bd80-d972af2c7d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:16.805135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409db2d-1e1d-4522-bd80-d972af2c7d98 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:16.805522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409db2d-1e1d-4522-bd80-d972af2c7d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:16.805834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409db2d-1e1d-4522-bd80-d972af2c7d98 None admin] PATH_INFO: `/v3/projects/96d7d51602334751ad658ced10495775` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:16.806943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409db2d-1e1d-4522-bd80-d972af2c7d98 None admin] RBAC: Authorizing `identity:get_project(project_id=96d7d51602334751ad658ced10495775)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:16.808364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409db2d-1e1d-4522-bd80-d972af2c7d98 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:16.810018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 598/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:32:16 2022] GET /identity/v3/projects/96d7d51602334751ad658ced10495775 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:16.815893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c7b78-435f-4107-9cda-3da1fac2eb53 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:16.834253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:16.912730 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c7b78-435f-4107-9cda-3da1fac2eb53 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:16.946408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1e298-086a-44af-af0a-4c6d826689fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:16.947047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1e298-086a-44af-af0a-4c6d826689fe None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:16.947323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1e298-086a-44af-af0a-4c6d826689fe None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:16.947594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1e298-086a-44af-af0a-4c6d826689fe None admin] PATH_INFO: `/v3/projects/96d7d51602334751ad658ced10495775/users/3bac8392931845efb0b44232806b2944/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:16.949749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1e298-086a-44af-af0a-4c6d826689fe None admin] RBAC: Authorizing `identity:create_grant(project_id=96d7d51602334751ad658ced10495775, user_id=3bac8392931845efb0b44232806b2944, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:16.951166 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1e298-086a-44af-af0a-4c6d826689fe None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:17.026597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 594/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:32:16 2022] PUT /identity/v3/projects/96d7d51602334751ad658ced10495775/users/3bac8392931845efb0b44232806b2944/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:32:19.163661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2482f56-7f14-4358-9233-fb8272b4e4f0 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:19.164185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2482f56-7f14-4358-9233-fb8272b4e4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:19.164185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2482f56-7f14-4358-9233-fb8272b4e4f0 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:19.165541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 599/1193] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:32:19.170704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ea043-a2ba-4f9e-8f35-d366516814a3 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:19.171108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ea043-a2ba-4f9e-8f35-d366516814a3 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:19.171250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ea043-a2ba-4f9e-8f35-d366516814a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:19.189559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b68ea043-a2ba-4f9e-8f35-d366516814a3 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:32:19.190780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b68ea043-a2ba-4f9e-8f35-d366516814a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:32:19.254825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 595/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:32:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:32:19.523303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2482f56-7f14-4358-9233-fb8272b4e4f0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:19.625385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2482f56-7f14-4358-9233-fb8272b4e4f0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:19.658169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b77b7-9c5b-4a3b-9df0-771b1cd2b959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:19.658775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b77b7-9c5b-4a3b-9df0-771b1cd2b959 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:19.659052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b77b7-9c5b-4a3b-9df0-771b1cd2b959 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:19.659320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b77b7-9c5b-4a3b-9df0-771b1cd2b959 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:19.661860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:19.669429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b77b7-9c5b-4a3b-9df0-771b1cd2b959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f45b77b7-9c5b-4a3b-9df0-771b1cd2b959 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:32:19.674173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:32:19.677135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:32:19.678223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 600/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:32:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:32:19.681623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ea043-a2ba-4f9e-8f35-d366516814a3 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:19.689548 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:19.789546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ea043-a2ba-4f9e-8f35-d366516814a3 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:19.823966 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93310b72-7899-43f4-be47-f4ae6a08c027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:19.824570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93310b72-7899-43f4-be47-f4ae6a08c027 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:19.824842 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93310b72-7899-43f4-be47-f4ae6a08c027 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:19.825107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93310b72-7899-43f4-be47-f4ae6a08c027 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:19.825780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93310b72-7899-43f4-be47-f4ae6a08c027 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:32:19.826078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93310b72-7899-43f4-be47-f4ae6a08c027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:19.827428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93310b72-7899-43f4-be47-f4ae6a08c027 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:19.835697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 596/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:32:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:19.841562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b77b7-9c5b-4a3b-9df0-771b1cd2b959 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:19.936515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b77b7-9c5b-4a3b-9df0-771b1cd2b959 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:19.969343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dc9af-0d2f-4055-9d4e-8383c9b44f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:19.969972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dc9af-0d2f-4055-9d4e-8383c9b44f90 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:19.970252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dc9af-0d2f-4055-9d4e-8383c9b44f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:19.970526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dc9af-0d2f-4055-9d4e-8383c9b44f90 None admin] PATH_INFO: `/v3/users/3bac8392931845efb0b44232806b2944` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:19.971645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dc9af-0d2f-4055-9d4e-8383c9b44f90 None admin] RBAC: Authorizing `identity:get_user(user_id=3bac8392931845efb0b44232806b2944)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:19.973056 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dc9af-0d2f-4055-9d4e-8383c9b44f90 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:19.974632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 601/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Sep 17 03:32:19 2022] GET /identity/v3/users/3bac8392931845efb0b44232806b2944 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:19.980281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93310b72-7899-43f4-be47-f4ae6a08c027 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:20.051767 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:20.086185 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93310b72-7899-43f4-be47-f4ae6a08c027 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:20.124019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da49be8-48dd-4440-8b1a-974a7381b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:20.124610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da49be8-48dd-4440-8b1a-974a7381b558 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:20.124882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da49be8-48dd-4440-8b1a-974a7381b558 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:20.125151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da49be8-48dd-4440-8b1a-974a7381b558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:20.189697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da49be8-48dd-4440-8b1a-974a7381b558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:20.191163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da49be8-48dd-4440-8b1a-974a7381b558 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:20.252738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:20.285516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 597/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:32:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:32:20.292284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dc9af-0d2f-4055-9d4e-8383c9b44f90 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:20.359459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:20.400206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dc9af-0d2f-4055-9d4e-8383c9b44f90 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:20.432772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea37d5d-7e40-454a-9119-7b0290f5f2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:20.433384 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea37d5d-7e40-454a-9119-7b0290f5f2ab None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:20.433660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea37d5d-7e40-454a-9119-7b0290f5f2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:20.433961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea37d5d-7e40-454a-9119-7b0290f5f2ab None admin] PATH_INFO: `/v3/projects/96d7d51602334751ad658ced10495775` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:20.435048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea37d5d-7e40-454a-9119-7b0290f5f2ab None admin] RBAC: Authorizing `identity:get_project(project_id=96d7d51602334751ad658ced10495775)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:20.436602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea37d5d-7e40-454a-9119-7b0290f5f2ab None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:20.438308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 602/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Sep 17 03:32:20 2022] GET /identity/v3/projects/96d7d51602334751ad658ced10495775 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:20.444722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da49be8-48dd-4440-8b1a-974a7381b558 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:32:20.539422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da49be8-48dd-4440-8b1a-974a7381b558 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:32:20.571638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05ff41-633b-4f88-bf02-9cbc06c1d654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:32:20.572218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05ff41-633b-4f88-bf02-9cbc06c1d654 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:32:20.572488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05ff41-633b-4f88-bf02-9cbc06c1d654 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:32:20.572753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05ff41-633b-4f88-bf02-9cbc06c1d654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:32:20.573455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05ff41-633b-4f88-bf02-9cbc06c1d654 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=96d7d51602334751ad658ced10495775, user.id=3bac8392931845efb0b44232806b2944) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:32:20.574035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05ff41-633b-4f88-bf02-9cbc06c1d654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:32:20.575446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05ff41-633b-4f88-bf02-9cbc06c1d654 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:32:20.580901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 598/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:32:20 2022] GET /identity/v3/role_assignments?user.id=3bac8392931845efb0b44232806b2944&scope.project.id=96d7d51602334751ad658ced10495775&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:32:30.428175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=34 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:32:30.578430 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=84 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:32.716551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30765273-b72e-4773-acc5-6225b60480c3 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:32.717131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30765273-b72e-4773-acc5-6225b60480c3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:32.717131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30765273-b72e-4773-acc5-6225b60480c3 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:32.718500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 603/1201] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:33:32.723789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979478c-3ccb-45d5-8a2b-0a85553a4ced None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:32.724229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979478c-3ccb-45d5-8a2b-0a85553a4ced None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:32.724328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979478c-3ccb-45d5-8a2b-0a85553a4ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:32.743623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-c979478c-3ccb-45d5-8a2b-0a85553a4ced None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:33:32.748519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c979478c-3ccb-45d5-8a2b-0a85553a4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:33:32.809540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 599/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:33:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:33:33.091558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30765273-b72e-4773-acc5-6225b60480c3 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:33.193700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30765273-b72e-4773-acc5-6225b60480c3 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:33.228172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6963ae3-4086-436e-aebb-5239231fbabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:33.228838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6963ae3-4086-436e-aebb-5239231fbabf None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:33.229115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6963ae3-4086-436e-aebb-5239231fbabf None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:33.229389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6963ae3-4086-436e-aebb-5239231fbabf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:33.239183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6963ae3-4086-436e-aebb-5239231fbabf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:33.240701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6963ae3-4086-436e-aebb-5239231fbabf None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d6963ae3-4086-436e-aebb-5239231fbabf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:33.256129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:33.257907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:33.259208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:33.259208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:33.259208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:33.259208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 604/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:33.264071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979478c-3ccb-45d5-8a2b-0a85553a4ced None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:33.360320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979478c-3ccb-45d5-8a2b-0a85553a4ced None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:33.388793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:33.394532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e65aeb-ed2c-4adb-9b89-087fa725da6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:33.395117 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65aeb-ed2c-4adb-9b89-087fa725da6a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:33.395391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65aeb-ed2c-4adb-9b89-087fa725da6a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:33.395658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65aeb-ed2c-4adb-9b89-087fa725da6a None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:33.396282 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65aeb-ed2c-4adb-9b89-087fa725da6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:33.396579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65aeb-ed2c-4adb-9b89-087fa725da6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:33.398030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65aeb-ed2c-4adb-9b89-087fa725da6a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:33.408820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 600/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:33.415175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6963ae3-4086-436e-aebb-5239231fbabf None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:33.467656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:33.513489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6963ae3-4086-436e-aebb-5239231fbabf None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:33.546234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ab131-e7c6-493c-8151-cebca849b431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:33.546818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ab131-e7c6-493c-8151-cebca849b431 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:33.547101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ab131-e7c6-493c-8151-cebca849b431 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:33.547379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ab131-e7c6-493c-8151-cebca849b431 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:33.548031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ab131-e7c6-493c-8151-cebca849b431 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:33.549435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ab131-e7c6-493c-8151-cebca849b431 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:33.688867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 605/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Sep 17 03:33:33 2022] POST /identity/v3/users => generated 251 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:33:35.857572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d45af2-1f06-4c06-93fe-c2949bf8e613 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:35.858179 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d45af2-1f06-4c06-93fe-c2949bf8e613 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:35.858179 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d45af2-1f06-4c06-93fe-c2949bf8e613 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:35.859493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 601/1206] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:33:35.864607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3b9ad-4264-482a-b366-eb317857c481 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:35.865046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3b9ad-4264-482a-b366-eb317857c481 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:35.865145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3b9ad-4264-482a-b366-eb317857c481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:35.883775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-01c3b9ad-4264-482a-b366-eb317857c481 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:33:35.884939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-01c3b9ad-4264-482a-b366-eb317857c481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:33:35.949278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 606/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:33:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:33:36.217811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d45af2-1f06-4c06-93fe-c2949bf8e613 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:36.316368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d45af2-1f06-4c06-93fe-c2949bf8e613 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:36.349526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eae40-7b8a-45a6-b903-d895cb6e1ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:36.350323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eae40-7b8a-45a6-b903-d895cb6e1ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:36.350792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eae40-7b8a-45a6-b903-d895cb6e1ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:36.351072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eae40-7b8a-45a6-b903-d895cb6e1ee7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:36.361282 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eae40-7b8a-45a6-b903-d895cb6e1ee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-501eae40-7b8a-45a6-b903-d895cb6e1ee7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:33:36.366731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:36.367808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:36.368701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 602/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:33:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:36.374164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3b9ad-4264-482a-b366-eb317857c481 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:36.420549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:36.479643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3b9ad-4264-482a-b366-eb317857c481 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:36.512201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2de325-4285-4f8a-bc4d-052b2a315a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:36.512820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2de325-4285-4f8a-bc4d-052b2a315a10 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:36.513096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2de325-4285-4f8a-bc4d-052b2a315a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:36.513364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2de325-4285-4f8a-bc4d-052b2a315a10 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:36.514080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2de325-4285-4f8a-bc4d-052b2a315a10 None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:36.514377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2de325-4285-4f8a-bc4d-052b2a315a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:36.515723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2de325-4285-4f8a-bc4d-052b2a315a10 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:36.523486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 607/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:33:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:36.529456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eae40-7b8a-45a6-b903-d895cb6e1ee7 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:36.579874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:36.630547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eae40-7b8a-45a6-b903-d895cb6e1ee7 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:36.669059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf08d54-06da-471e-b2cd-e77e0bd71ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:36.669694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf08d54-06da-471e-b2cd-e77e0bd71ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:36.669995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf08d54-06da-471e-b2cd-e77e0bd71ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:36.676714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf08d54-06da-471e-b2cd-e77e0bd71ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:36.688531 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf08d54-06da-471e-b2cd-e77e0bd71ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:36.689991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf08d54-06da-471e-b2cd-e77e0bd71ec6 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-8bf08d54-06da-471e-b2cd-e77e0bd71ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:36.704275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:36.705494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:36.706836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:36.706836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:36.706836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:36.706836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 603/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:36.711309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2de325-4285-4f8a-bc4d-052b2a315a10 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:36.811389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:36.814248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2de325-4285-4f8a-bc4d-052b2a315a10 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:36.847712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a0c6-10ce-4bc6-aee3-56650c37b633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:36.848294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a0c6-10ce-4bc6-aee3-56650c37b633 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:36.848570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a0c6-10ce-4bc6-aee3-56650c37b633 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:36.848841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a0c6-10ce-4bc6-aee3-56650c37b633 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:36.849472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a0c6-10ce-4bc6-aee3-56650c37b633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:36.849798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a0c6-10ce-4bc6-aee3-56650c37b633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:36.851256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a0c6-10ce-4bc6-aee3-56650c37b633 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:36.862247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 608/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:36.868757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf08d54-06da-471e-b2cd-e77e0bd71ec6 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:36.968209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf08d54-06da-471e-b2cd-e77e0bd71ec6 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:36.969681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:37.002087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52507039-a592-4b2a-a83a-85de958e986e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:37.002709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52507039-a592-4b2a-a83a-85de958e986e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:37.002989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52507039-a592-4b2a-a83a-85de958e986e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:37.003262 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52507039-a592-4b2a-a83a-85de958e986e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:37.011950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52507039-a592-4b2a-a83a-85de958e986e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:37.013415 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52507039-a592-4b2a-a83a-85de958e986e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-52507039-a592-4b2a-a83a-85de958e986e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:33:37.030319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:37.031383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:37.032481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 604/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:37.038470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a0c6-10ce-4bc6-aee3-56650c37b633 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:37.142779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a0c6-10ce-4bc6-aee3-56650c37b633 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:37.175577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ab00c-9b2b-4045-a688-299cd4dfd318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:37.176173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ab00c-9b2b-4045-a688-299cd4dfd318 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:37.176506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ab00c-9b2b-4045-a688-299cd4dfd318 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:37.176786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ab00c-9b2b-4045-a688-299cd4dfd318 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:37.177504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ab00c-9b2b-4045-a688-299cd4dfd318 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:37.177840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ab00c-9b2b-4045-a688-299cd4dfd318 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:37.179378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ab00c-9b2b-4045-a688-299cd4dfd318 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:37.196940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 609/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Sep 17 03:33:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:37.203185 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52507039-a592-4b2a-a83a-85de958e986e None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:37.302255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52507039-a592-4b2a-a83a-85de958e986e None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:37.343845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4059e4b-e986-4bbf-8082-92f9d9e2f027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:37.344103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4059e4b-e986-4bbf-8082-92f9d9e2f027 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:37.344299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4059e4b-e986-4bbf-8082-92f9d9e2f027 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:37.344571 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4059e4b-e986-4bbf-8082-92f9d9e2f027 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:37.378410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:37.415634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4059e4b-e986-4bbf-8082-92f9d9e2f027 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:37.417306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4059e4b-e986-4bbf-8082-92f9d9e2f027 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:37.513631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 605/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:33:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:33:37.519578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ab00c-9b2b-4045-a688-299cd4dfd318 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:37.539295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:37.618490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ab00c-9b2b-4045-a688-299cd4dfd318 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:37.653045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eed80f-b291-4d87-836b-79639a3edf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:37.653674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eed80f-b291-4d87-836b-79639a3edf14 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:37.653993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eed80f-b291-4d87-836b-79639a3edf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:37.654268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eed80f-b291-4d87-836b-79639a3edf14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:37.668160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eed80f-b291-4d87-836b-79639a3edf14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:37.669550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eed80f-b291-4d87-836b-79639a3edf14 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-98eed80f-b291-4d87-836b-79639a3edf14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:37.684605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:37.685675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:37.692747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:37.692747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:37.692747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:37.692747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 610/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:37.693050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4059e4b-e986-4bbf-8082-92f9d9e2f027 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:37.757010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:37.793303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4059e4b-e986-4bbf-8082-92f9d9e2f027 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:37.827445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238e7d0-de16-46ab-aede-1b366705f3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:37.828040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238e7d0-de16-46ab-aede-1b366705f3cc None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:37.828313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238e7d0-de16-46ab-aede-1b366705f3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:37.828581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238e7d0-de16-46ab-aede-1b366705f3cc None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:37.829207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238e7d0-de16-46ab-aede-1b366705f3cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:37.829508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238e7d0-de16-46ab-aede-1b366705f3cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:37.830902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238e7d0-de16-46ab-aede-1b366705f3cc None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:37.841673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 606/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:37.847397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eed80f-b291-4d87-836b-79639a3edf14 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:37.915855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:37.946059 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eed80f-b291-4d87-836b-79639a3edf14 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:37.980629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c62112-2e39-40d9-b790-a2ca57995c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:37.981282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c62112-2e39-40d9-b790-a2ca57995c07 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:37.981540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c62112-2e39-40d9-b790-a2ca57995c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:37.981846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c62112-2e39-40d9-b790-a2ca57995c07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:37.988591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c62112-2e39-40d9-b790-a2ca57995c07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:37.990055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c62112-2e39-40d9-b790-a2ca57995c07 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-40c62112-2e39-40d9-b790-a2ca57995c07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:33:38.001915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:38.005301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:38.006604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:33:38.006604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:33:38.006604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:33:38.006604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:33:38.006604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:33:38.006604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:33:38.006604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:33:38.006604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:38.006604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 611/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:33:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:38.009129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238e7d0-de16-46ab-aede-1b366705f3cc None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:38.109603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238e7d0-de16-46ab-aede-1b366705f3cc None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:38.131552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:38.142579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22fc07-597e-4e52-b9d2-f8063e99395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:38.143210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22fc07-597e-4e52-b9d2-f8063e99395f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:38.143483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22fc07-597e-4e52-b9d2-f8063e99395f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:38.143751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22fc07-597e-4e52-b9d2-f8063e99395f None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:38.144430 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22fc07-597e-4e52-b9d2-f8063e99395f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:38.144742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22fc07-597e-4e52-b9d2-f8063e99395f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:38.146211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22fc07-597e-4e52-b9d2-f8063e99395f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:38.157843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 607/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:33:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:38.164832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c62112-2e39-40d9-b790-a2ca57995c07 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:38.265290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c62112-2e39-40d9-b790-a2ca57995c07 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:38.296182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:38.301957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c301463-b997-4626-b9ac-0ca195ea3daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:38.302551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c301463-b997-4626-b9ac-0ca195ea3daa None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:38.302832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c301463-b997-4626-b9ac-0ca195ea3daa None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:38.303645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c301463-b997-4626-b9ac-0ca195ea3daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:38.304376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c301463-b997-4626-b9ac-0ca195ea3daa None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=34ec5c9c71d14762a4869b7d66e87ee3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:38.304796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c301463-b997-4626-b9ac-0ca195ea3daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:38.306249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c301463-b997-4626-b9ac-0ca195ea3daa None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:38.311696 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 612/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:33:38 2022] GET /identity/v3/role_assignments?user.id=34ec5c9c71d14762a4869b7d66e87ee3&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:33:40.543344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb9d7a-f4f7-4a97-b091-5d0650b73878 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:40.543885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb9d7a-f4f7-4a97-b091-5d0650b73878 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:40.543885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb9d7a-f4f7-4a97-b091-5d0650b73878 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:40.545271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 608/1220] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:33:40.550374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce46d9-01d8-4ea1-a196-dddbcff49b1d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:40.550743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce46d9-01d8-4ea1-a196-dddbcff49b1d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:40.550932 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce46d9-01d8-4ea1-a196-dddbcff49b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:40.570451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-25ce46d9-01d8-4ea1-a196-dddbcff49b1d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:33:40.571630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-25ce46d9-01d8-4ea1-a196-dddbcff49b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:33:40.638069 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 613/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:33:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:33:40.917014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb9d7a-f4f7-4a97-b091-5d0650b73878 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:41.019207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb9d7a-f4f7-4a97-b091-5d0650b73878 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:41.051689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55e17f-4fa2-414c-8dc7-7b3c1e1d73a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:41.052298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55e17f-4fa2-414c-8dc7-7b3c1e1d73a1 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:41.052580 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55e17f-4fa2-414c-8dc7-7b3c1e1d73a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:41.052845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55e17f-4fa2-414c-8dc7-7b3c1e1d73a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:41.063263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55e17f-4fa2-414c-8dc7-7b3c1e1d73a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-6a55e17f-4fa2-414c-8dc7-7b3c1e1d73a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:33:41.067920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:41.071039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:41.071927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 609/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:33:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:41.075586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce46d9-01d8-4ea1-a196-dddbcff49b1d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:41.172153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce46d9-01d8-4ea1-a196-dddbcff49b1d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:41.195427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:41.205971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06a57b-bd8e-4fde-8cc9-77ae1573a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:41.206570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06a57b-bd8e-4fde-8cc9-77ae1573a00f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:41.206851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06a57b-bd8e-4fde-8cc9-77ae1573a00f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:41.207126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06a57b-bd8e-4fde-8cc9-77ae1573a00f None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:41.207783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06a57b-bd8e-4fde-8cc9-77ae1573a00f None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:41.208086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06a57b-bd8e-4fde-8cc9-77ae1573a00f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:41.210159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06a57b-bd8e-4fde-8cc9-77ae1573a00f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:41.217831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 614/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:33:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:41.223723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55e17f-4fa2-414c-8dc7-7b3c1e1d73a1 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:41.268539 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:41.323448 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55e17f-4fa2-414c-8dc7-7b3c1e1d73a1 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:41.356771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2b1d3-4c3c-447d-abe6-2d70d18eebfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:41.357395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2b1d3-4c3c-447d-abe6-2d70d18eebfc None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:41.357671 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2b1d3-4c3c-447d-abe6-2d70d18eebfc None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:41.357971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2b1d3-4c3c-447d-abe6-2d70d18eebfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:41.369119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2b1d3-4c3c-447d-abe6-2d70d18eebfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:41.370608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2b1d3-4c3c-447d-abe6-2d70d18eebfc None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d4f2b1d3-4c3c-447d-abe6-2d70d18eebfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:41.385488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:41.386627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:41.387701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:41.387701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:41.387701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:41.387701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 610/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:41.392389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06a57b-bd8e-4fde-8cc9-77ae1573a00f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:41.489910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06a57b-bd8e-4fde-8cc9-77ae1573a00f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:41.522555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c488f-8ae8-49bd-8de8-11dbf84241f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:41.523164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c488f-8ae8-49bd-8de8-11dbf84241f1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:41.523436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c488f-8ae8-49bd-8de8-11dbf84241f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:41.523703 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c488f-8ae8-49bd-8de8-11dbf84241f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:41.524331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c488f-8ae8-49bd-8de8-11dbf84241f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:41.524626 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c488f-8ae8-49bd-8de8-11dbf84241f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:41.526007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c488f-8ae8-49bd-8de8-11dbf84241f1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:41.538085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 615/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:41.543837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2b1d3-4c3c-447d-abe6-2d70d18eebfc None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:41.635804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:41.641987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2b1d3-4c3c-447d-abe6-2d70d18eebfc None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:41.674497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a59dc6-758c-4501-bc09-f7dc308020b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:41.675109 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a59dc6-758c-4501-bc09-f7dc308020b0 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:41.675383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a59dc6-758c-4501-bc09-f7dc308020b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:41.675651 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a59dc6-758c-4501-bc09-f7dc308020b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:41.682887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a59dc6-758c-4501-bc09-f7dc308020b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:41.684309 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a59dc6-758c-4501-bc09-f7dc308020b0 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-86a59dc6-758c-4501-bc09-f7dc308020b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:33:41.695389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:41.702631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:41.703704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 611/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:41.704438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c488f-8ae8-49bd-8de8-11dbf84241f1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:41.749595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:41.804115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c488f-8ae8-49bd-8de8-11dbf84241f1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:41.838422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151c247-bdb7-4565-af46-52a7e8bc9f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:41.839012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151c247-bdb7-4565-af46-52a7e8bc9f5e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:41.839339 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151c247-bdb7-4565-af46-52a7e8bc9f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:41.839609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151c247-bdb7-4565-af46-52a7e8bc9f5e None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:41.840371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151c247-bdb7-4565-af46-52a7e8bc9f5e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:41.840675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151c247-bdb7-4565-af46-52a7e8bc9f5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:41.842118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151c247-bdb7-4565-af46-52a7e8bc9f5e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:41.859104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 616/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Sep 17 03:33:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:41.865292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a59dc6-758c-4501-bc09-f7dc308020b0 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:41.963221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a59dc6-758c-4501-bc09-f7dc308020b0 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:41.995554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f520c-afde-41e5-90cf-25a7c278dea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:41.996302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f520c-afde-41e5-90cf-25a7c278dea5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:41.996602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f520c-afde-41e5-90cf-25a7c278dea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:41.996869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f520c-afde-41e5-90cf-25a7c278dea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:42.008522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f520c-afde-41e5-90cf-25a7c278dea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:42.009993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f520c-afde-41e5-90cf-25a7c278dea5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-495f520c-afde-41e5-90cf-25a7c278dea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:42.024846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:42.026054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:42.027030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:42.027030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:42.027030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:42.027030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 612/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:42.031688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151c247-bdb7-4565-af46-52a7e8bc9f5e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:42.134241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151c247-bdb7-4565-af46-52a7e8bc9f5e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:42.136529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:42.166718 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce9d9a-f3e1-4a94-9c78-30a1f819edda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:42.167343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce9d9a-f3e1-4a94-9c78-30a1f819edda None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:42.167653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce9d9a-f3e1-4a94-9c78-30a1f819edda None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:42.167933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce9d9a-f3e1-4a94-9c78-30a1f819edda None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:42.168558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce9d9a-f3e1-4a94-9c78-30a1f819edda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:42.168857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce9d9a-f3e1-4a94-9c78-30a1f819edda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:42.170239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce9d9a-f3e1-4a94-9c78-30a1f819edda None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:42.182488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 617/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:42.188155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f520c-afde-41e5-90cf-25a7c278dea5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:42.197988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:42.295220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f520c-afde-41e5-90cf-25a7c278dea5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:42.329987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a605b-183f-4303-9a8f-e0ef3aa5e15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:42.330637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a605b-183f-4303-9a8f-e0ef3aa5e15f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:42.330914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a605b-183f-4303-9a8f-e0ef3aa5e15f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:42.331186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a605b-183f-4303-9a8f-e0ef3aa5e15f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:42.337788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a605b-183f-4303-9a8f-e0ef3aa5e15f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:42.339222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a605b-183f-4303-9a8f-e0ef3aa5e15f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-2a2a605b-183f-4303-9a8f-e0ef3aa5e15f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:33:42.350495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:42.351526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:42.352813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:33:42.352813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:33:42.352813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:33:42.352813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:33:42.352813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:33:42.352813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:33:42.352813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:33:42.352813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:42.352813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 613/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:42.357498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce9d9a-f3e1-4a94-9c78-30a1f819edda None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:42.459559 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce9d9a-f3e1-4a94-9c78-30a1f819edda None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:42.493548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467f2f-8933-4301-9911-330032fd9cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:42.494181 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467f2f-8933-4301-9911-330032fd9cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:42.494462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467f2f-8933-4301-9911-330032fd9cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:42.494736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467f2f-8933-4301-9911-330032fd9cc1 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:42.495418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467f2f-8933-4301-9911-330032fd9cc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:42.495723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467f2f-8933-4301-9911-330032fd9cc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:42.497192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467f2f-8933-4301-9911-330032fd9cc1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:42.509077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 618/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:33:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:42.515203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a605b-183f-4303-9a8f-e0ef3aa5e15f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:42.580426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:42.613802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a605b-183f-4303-9a8f-e0ef3aa5e15f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:42.645849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19552d55-0fe7-4823-8d6e-3fe1479a5637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:42.646487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19552d55-0fe7-4823-8d6e-3fe1479a5637 None admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:42.647356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19552d55-0fe7-4823-8d6e-3fe1479a5637 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:42.647640 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19552d55-0fe7-4823-8d6e-3fe1479a5637 None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/34ec5c9c71d14762a4869b7d66e87ee3/roles/e33e2d7e93b4409fb2c449241a219d0d` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:42.661085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19552d55-0fe7-4823-8d6e-3fe1479a5637 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=34ec5c9c71d14762a4869b7d66e87ee3, role_id=e33e2d7e93b4409fb2c449241a219d0d)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:42.662566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19552d55-0fe7-4823-8d6e-3fe1479a5637 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:42.746335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 614/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:33:42 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/34ec5c9c71d14762a4869b7d66e87ee3/roles/e33e2d7e93b4409fb2c449241a219d0d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:33:44.886099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d18aa-10c3-4783-af5e-e240098af9aa None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:44.886641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d18aa-10c3-4783-af5e-e240098af9aa None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:44.886738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d18aa-10c3-4783-af5e-e240098af9aa None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:44.888055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 619/1233] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:33:44.893427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39996525-9ce4-47e1-b600-a6421e8fb1c7 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:44.893908 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39996525-9ce4-47e1-b600-a6421e8fb1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:44.899494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39996525-9ce4-47e1-b600-a6421e8fb1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:44.912860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-39996525-9ce4-47e1-b600-a6421e8fb1c7 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:33:44.914084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-39996525-9ce4-47e1-b600-a6421e8fb1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:33:44.980377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 615/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:33:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 03:33:45.252155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d18aa-10c3-4783-af5e-e240098af9aa None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:45.280608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:45.364448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d18aa-10c3-4783-af5e-e240098af9aa None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:45.402155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae074ca-1f95-4821-895e-d27e4026f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:45.402783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae074ca-1f95-4821-895e-d27e4026f35a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:45.403061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae074ca-1f95-4821-895e-d27e4026f35a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:45.403331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae074ca-1f95-4821-895e-d27e4026f35a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:45.414382 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae074ca-1f95-4821-895e-d27e4026f35a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-8ae074ca-1f95-4821-895e-d27e4026f35a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:33:45.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:45.420238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:45.421451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 620/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Sep 17 03:33:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:45.426534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39996525-9ce4-47e1-b600-a6421e8fb1c7 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:45.476132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:45.534613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39996525-9ce4-47e1-b600-a6421e8fb1c7 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:45.570259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950547d7-b872-4f18-b065-dc1d60034519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:45.570954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950547d7-b872-4f18-b065-dc1d60034519 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:45.571236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950547d7-b872-4f18-b065-dc1d60034519 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:45.571506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950547d7-b872-4f18-b065-dc1d60034519 None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:45.572192 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950547d7-b872-4f18-b065-dc1d60034519 None admin] RBAC: Adding query filter params (name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:45.572488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950547d7-b872-4f18-b065-dc1d60034519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:45.574134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950547d7-b872-4f18-b065-dc1d60034519 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:45.582071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 616/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Sep 17 03:33:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:45.588068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae074ca-1f95-4821-895e-d27e4026f35a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:45.675731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:45.690749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae074ca-1f95-4821-895e-d27e4026f35a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:45.723831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754bbc0-458e-45c2-9d3e-3ee149d2b4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:45.724440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754bbc0-458e-45c2-9d3e-3ee149d2b4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:45.724759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754bbc0-458e-45c2-9d3e-3ee149d2b4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:45.725180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754bbc0-458e-45c2-9d3e-3ee149d2b4b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:45.734730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754bbc0-458e-45c2-9d3e-3ee149d2b4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:45.736158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754bbc0-458e-45c2-9d3e-3ee149d2b4b9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c754bbc0-458e-45c2-9d3e-3ee149d2b4b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:45.750124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:45.751280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:45.752378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:45.752378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:45.752378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:45.752378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 621/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:45.757761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950547d7-b872-4f18-b065-dc1d60034519 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:45.857015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950547d7-b872-4f18-b065-dc1d60034519 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:45.866398 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:45.890766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcd5bb-0c01-4282-83e3-3506fbe54c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:45.891367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd5bb-0c01-4282-83e3-3506fbe54c8f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:45.891646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd5bb-0c01-4282-83e3-3506fbe54c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:45.891918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd5bb-0c01-4282-83e3-3506fbe54c8f None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:45.892554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcd5bb-0c01-4282-83e3-3506fbe54c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:45.892856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcd5bb-0c01-4282-83e3-3506fbe54c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:45.894309 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcd5bb-0c01-4282-83e3-3506fbe54c8f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:45.905589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 617/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:45.911556 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754bbc0-458e-45c2-9d3e-3ee149d2b4b9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:46.011691 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754bbc0-458e-45c2-9d3e-3ee149d2b4b9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:46.045909 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e65bc4-cee7-4bb5-81c6-6573ba381964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:46.046530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e65bc4-cee7-4bb5-81c6-6573ba381964 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:46.046867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e65bc4-cee7-4bb5-81c6-6573ba381964 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:46.047149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e65bc4-cee7-4bb5-81c6-6573ba381964 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:46.049846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:46.055739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e65bc4-cee7-4bb5-81c6-6573ba381964 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:46.057076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e65bc4-cee7-4bb5-81c6-6573ba381964 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-83e65bc4-cee7-4bb5-81c6-6573ba381964 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:33:46.069484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:46.070655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:46.072008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 622/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:46.076809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcd5bb-0c01-4282-83e3-3506fbe54c8f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:46.175809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcd5bb-0c01-4282-83e3-3506fbe54c8f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:46.209130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed4452-e1e8-4af6-b984-d49da0cd273a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:46.209784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4452-e1e8-4af6-b984-d49da0cd273a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:46.210005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4452-e1e8-4af6-b984-d49da0cd273a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:46.210271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4452-e1e8-4af6-b984-d49da0cd273a None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:46.210972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed4452-e1e8-4af6-b984-d49da0cd273a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:46.211294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed4452-e1e8-4af6-b984-d49da0cd273a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:46.212806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed4452-e1e8-4af6-b984-d49da0cd273a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:46.230303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 618/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Sep 17 03:33:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:46.236089 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e65bc4-cee7-4bb5-81c6-6573ba381964 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:46.335146 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e65bc4-cee7-4bb5-81c6-6573ba381964 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:46.368217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492619b2-fbf9-4220-90a8-944d2082df8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:46.371431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492619b2-fbf9-4220-90a8-944d2082df8e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:46.371431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492619b2-fbf9-4220-90a8-944d2082df8e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:46.371431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492619b2-fbf9-4220-90a8-944d2082df8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:46.427733 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:46.434673 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492619b2-fbf9-4220-90a8-944d2082df8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:46.436163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492619b2-fbf9-4220-90a8-944d2082df8e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:46.540247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 623/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:33:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:33:46.546259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed4452-e1e8-4af6-b984-d49da0cd273a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:46.574220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:46.645114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed4452-e1e8-4af6-b984-d49da0cd273a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:46.679315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f02e3-1f5b-411b-8b4f-efb170551fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:46.679935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f02e3-1f5b-411b-8b4f-efb170551fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:46.680209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f02e3-1f5b-411b-8b4f-efb170551fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:46.680475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f02e3-1f5b-411b-8b4f-efb170551fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:46.690074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f02e3-1f5b-411b-8b4f-efb170551fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:46.691503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f02e3-1f5b-411b-8b4f-efb170551fe6 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-407f02e3-1f5b-411b-8b4f-efb170551fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:46.707264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:46.708476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:46.715342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:46.715342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:46.715342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:46.715342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 619/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:33:46.715628 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492619b2-fbf9-4220-90a8-944d2082df8e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:46.807134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:46.812676 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492619b2-fbf9-4220-90a8-944d2082df8e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:46.845840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053be3c5-63a0-4d24-917e-f6768311cf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:46.846440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053be3c5-63a0-4d24-917e-f6768311cf8e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:46.846721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053be3c5-63a0-4d24-917e-f6768311cf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:46.846993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053be3c5-63a0-4d24-917e-f6768311cf8e None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:46.847625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053be3c5-63a0-4d24-917e-f6768311cf8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:46.847925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053be3c5-63a0-4d24-917e-f6768311cf8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:46.849264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053be3c5-63a0-4d24-917e-f6768311cf8e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:46.860596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 624/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:46.867124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f02e3-1f5b-411b-8b4f-efb170551fe6 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:46.945526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:46.966838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f02e3-1f5b-411b-8b4f-efb170551fe6 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:47.000072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920851a8-7e15-4fbf-9916-8592672a28e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:47.000686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920851a8-7e15-4fbf-9916-8592672a28e1 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:47.000961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920851a8-7e15-4fbf-9916-8592672a28e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:47.002887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920851a8-7e15-4fbf-9916-8592672a28e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:47.008060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920851a8-7e15-4fbf-9916-8592672a28e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:47.009483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920851a8-7e15-4fbf-9916-8592672a28e1 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-920851a8-7e15-4fbf-9916-8592672a28e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:33:47.021647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:47.030286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:47.031400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:33:47.031400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:33:47.031400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:33:47.031400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:33:47.031400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:33:47.031400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:33:47.031400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:33:47.031400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:47.031400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 620/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:33:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 03:33:47.031992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053be3c5-63a0-4d24-917e-f6768311cf8e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:47.126647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053be3c5-63a0-4d24-917e-f6768311cf8e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:47.159190 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c256d-482c-4cef-952f-019691ee1f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:47.159786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c256d-482c-4cef-952f-019691ee1f78 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:47.160065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c256d-482c-4cef-952f-019691ee1f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:47.160339 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c256d-482c-4cef-952f-019691ee1f78 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:47.161017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c256d-482c-4cef-952f-019691ee1f78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:47.161322 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c256d-482c-4cef-952f-019691ee1f78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:47.163403 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c256d-482c-4cef-952f-019691ee1f78 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:47.175036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 625/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:47.181825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920851a8-7e15-4fbf-9916-8592672a28e1 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:47.281741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920851a8-7e15-4fbf-9916-8592672a28e1 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:47.315133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc06f21-7572-4acd-9124-bd4a96dcc6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:47.315719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc06f21-7572-4acd-9124-bd4a96dcc6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:47.315993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc06f21-7572-4acd-9124-bd4a96dcc6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:47.316259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc06f21-7572-4acd-9124-bd4a96dcc6b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:47.316970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc06f21-7572-4acd-9124-bd4a96dcc6b3 None admin] RBAC: Adding query filter params (role.id=e33e2d7e93b4409fb2c449241a219d0d, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=34ec5c9c71d14762a4869b7d66e87ee3) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:47.317277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc06f21-7572-4acd-9124-bd4a96dcc6b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:47.318973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc06f21-7572-4acd-9124-bd4a96dcc6b3 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:47.324854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 621/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:33:47 2022] GET /identity/v3/role_assignments?user.id=34ec5c9c71d14762a4869b7d66e87ee3&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=e33e2d7e93b4409fb2c449241a219d0d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:49.614540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b01e2e-17c6-4283-ab17-6663a33ed77e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:49.615093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b01e2e-17c6-4283-ab17-6663a33ed77e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:49.615093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b01e2e-17c6-4283-ab17-6663a33ed77e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:49.617577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 626/1247] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:33:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:33:49.622931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050d358-bb11-44e9-a93d-52b004aac28d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:49.623323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050d358-bb11-44e9-a93d-52b004aac28d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:49.623536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050d358-bb11-44e9-a93d-52b004aac28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:49.629801 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:49.642491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-c050d358-bb11-44e9-a93d-52b004aac28d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:33:49.643676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c050d358-bb11-44e9-a93d-52b004aac28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:33:49.709271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 622/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:33:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:33:49.978874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b01e2e-17c6-4283-ab17-6663a33ed77e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:49.997554 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:50.079931 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b01e2e-17c6-4283-ab17-6663a33ed77e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:50.112751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04c3b3-f607-48df-958d-57f07c54e95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:50.113370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04c3b3-f607-48df-958d-57f07c54e95c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:50.113653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04c3b3-f607-48df-958d-57f07c54e95c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:50.113960 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04c3b3-f607-48df-958d-57f07c54e95c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:50.124700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04c3b3-f607-48df-958d-57f07c54e95c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3b04c3b3-f607-48df-958d-57f07c54e95c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:33:50.129447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:50.130511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:50.131860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 627/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:33:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:50.136869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050d358-bb11-44e9-a93d-52b004aac28d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:50.233824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050d358-bb11-44e9-a93d-52b004aac28d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:50.262800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:50.267633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331d728-29e1-487b-837e-2e27eb91452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:50.268230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d728-29e1-487b-837e-2e27eb91452c None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:50.268503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d728-29e1-487b-837e-2e27eb91452c None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:50.268771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d728-29e1-487b-837e-2e27eb91452c None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:50.269424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331d728-29e1-487b-837e-2e27eb91452c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:50.269744 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331d728-29e1-487b-837e-2e27eb91452c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:50.271162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331d728-29e1-487b-837e-2e27eb91452c None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:50.279105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 623/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:33:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:50.285064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04c3b3-f607-48df-958d-57f07c54e95c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:50.360768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:50.383680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04c3b3-f607-48df-958d-57f07c54e95c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:50.415754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a7151-80dc-4f2e-a9e6-38de5a45d5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:50.416379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a7151-80dc-4f2e-a9e6-38de5a45d5bd None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:50.416688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a7151-80dc-4f2e-a9e6-38de5a45d5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:50.416964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a7151-80dc-4f2e-a9e6-38de5a45d5bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:50.426491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a7151-80dc-4f2e-a9e6-38de5a45d5bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:50.428535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a7151-80dc-4f2e-a9e6-38de5a45d5bd None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c61a7151-80dc-4f2e-a9e6-38de5a45d5bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:50.442546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:50.443798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:50.444725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:50.444725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:50.444725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:50.444725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 628/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:50.450677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331d728-29e1-487b-837e-2e27eb91452c None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:50.551535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331d728-29e1-487b-837e-2e27eb91452c None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:50.585890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8ceca-dbaf-48d9-a57e-17487d6ddf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:50.586493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8ceca-dbaf-48d9-a57e-17487d6ddf2b None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:50.586771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8ceca-dbaf-48d9-a57e-17487d6ddf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:50.587045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8ceca-dbaf-48d9-a57e-17487d6ddf2b None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:50.587678 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8ceca-dbaf-48d9-a57e-17487d6ddf2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:50.587978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8ceca-dbaf-48d9-a57e-17487d6ddf2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:50.589355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8ceca-dbaf-48d9-a57e-17487d6ddf2b None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:50.601041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 624/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:50.607048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a7151-80dc-4f2e-a9e6-38de5a45d5bd None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:50.707649 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a7151-80dc-4f2e-a9e6-38de5a45d5bd None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:50.733712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:50.741858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431653ae-b0bc-4083-a9bb-9e7a5643a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:50.742474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431653ae-b0bc-4083-a9bb-9e7a5643a065 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:50.742750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431653ae-b0bc-4083-a9bb-9e7a5643a065 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:50.743021 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431653ae-b0bc-4083-a9bb-9e7a5643a065 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:50.750158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431653ae-b0bc-4083-a9bb-9e7a5643a065 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:50.751590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431653ae-b0bc-4083-a9bb-9e7a5643a065 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-431653ae-b0bc-4083-a9bb-9e7a5643a065 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:33:50.762638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:50.765527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:50.766769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 629/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:50.769847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8ceca-dbaf-48d9-a57e-17487d6ddf2b None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:50.826781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:50.870785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8ceca-dbaf-48d9-a57e-17487d6ddf2b None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:50.904451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c69529-0254-4ee5-8a90-f0173ec13048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:50.905030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c69529-0254-4ee5-8a90-f0173ec13048 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:50.905301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c69529-0254-4ee5-8a90-f0173ec13048 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:50.905568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c69529-0254-4ee5-8a90-f0173ec13048 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:50.906304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c69529-0254-4ee5-8a90-f0173ec13048 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:50.906605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c69529-0254-4ee5-8a90-f0173ec13048 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:50.908108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c69529-0254-4ee5-8a90-f0173ec13048 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:50.926212 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 625/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Sep 17 03:33:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:50.932137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431653ae-b0bc-4083-a9bb-9e7a5643a065 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:51.034124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431653ae-b0bc-4083-a9bb-9e7a5643a065 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:51.066521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817e2cd-32d2-41ba-b88b-aa32b77025c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:51.067117 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817e2cd-32d2-41ba-b88b-aa32b77025c4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:51.067396 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817e2cd-32d2-41ba-b88b-aa32b77025c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:51.067666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817e2cd-32d2-41ba-b88b-aa32b77025c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:51.122643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:51.135083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817e2cd-32d2-41ba-b88b-aa32b77025c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:51.136523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817e2cd-32d2-41ba-b88b-aa32b77025c4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:51.231546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 630/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:33:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:33:51.237479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c69529-0254-4ee5-8a90-f0173ec13048 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:51.339835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c69529-0254-4ee5-8a90-f0173ec13048 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:51.354072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:51.374767 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7cbc8-bd5d-4771-837d-3ace9cd99680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:51.375320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7cbc8-bd5d-4771-837d-3ace9cd99680 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:51.375591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7cbc8-bd5d-4771-837d-3ace9cd99680 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:51.375855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7cbc8-bd5d-4771-837d-3ace9cd99680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:51.385961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7cbc8-bd5d-4771-837d-3ace9cd99680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:51.387385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7cbc8-bd5d-4771-837d-3ace9cd99680 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-eed7cbc8-bd5d-4771-837d-3ace9cd99680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:51.402089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:51.403261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:51.404408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:51.404408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:51.404408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:51.407859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 626/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:51.409638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817e2cd-32d2-41ba-b88b-aa32b77025c4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:51.495294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:51.507429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817e2cd-32d2-41ba-b88b-aa32b77025c4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:51.540103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c497eb0-4118-4b5c-8f2a-672be6eafb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:51.540737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c497eb0-4118-4b5c-8f2a-672be6eafb9e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:51.541000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c497eb0-4118-4b5c-8f2a-672be6eafb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:51.541269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c497eb0-4118-4b5c-8f2a-672be6eafb9e None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:51.541958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c497eb0-4118-4b5c-8f2a-672be6eafb9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:51.542256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c497eb0-4118-4b5c-8f2a-672be6eafb9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:51.543599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c497eb0-4118-4b5c-8f2a-672be6eafb9e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:51.554461 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 631/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:51.560181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7cbc8-bd5d-4771-837d-3ace9cd99680 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:51.660040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7cbc8-bd5d-4771-837d-3ace9cd99680 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:51.693153 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3813c7-b4f5-493d-a0e9-62aeeceb9a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:51.694778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3813c7-b4f5-493d-a0e9-62aeeceb9a3a None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:51.695056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3813c7-b4f5-493d-a0e9-62aeeceb9a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:51.695332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3813c7-b4f5-493d-a0e9-62aeeceb9a3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:51.702062 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3813c7-b4f5-493d-a0e9-62aeeceb9a3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:51.703528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3813c7-b4f5-493d-a0e9-62aeeceb9a3a None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-6d3813c7-b4f5-493d-a0e9-62aeeceb9a3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:33:51.715855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:51.716944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:51.718108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:33:51.718108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:33:51.718108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:33:51.718108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:33:51.718108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:33:51.718108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:33:51.718108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:33:51.718108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:51.718108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 627/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:33:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:51.722957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c497eb0-4118-4b5c-8f2a-672be6eafb9e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:51.824295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c497eb0-4118-4b5c-8f2a-672be6eafb9e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:51.856721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a82610-bfec-46a6-99fa-c3ee9da3b367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:51.857318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a82610-bfec-46a6-99fa-c3ee9da3b367 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:51.857591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a82610-bfec-46a6-99fa-c3ee9da3b367 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:51.857892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a82610-bfec-46a6-99fa-c3ee9da3b367 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:51.858581 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a82610-bfec-46a6-99fa-c3ee9da3b367 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:51.858908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a82610-bfec-46a6-99fa-c3ee9da3b367 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:51.860321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a82610-bfec-46a6-99fa-c3ee9da3b367 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:51.873272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 632/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:51.879587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3813c7-b4f5-493d-a0e9-62aeeceb9a3a None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:51.906492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:51.978520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3813c7-b4f5-493d-a0e9-62aeeceb9a3a None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:52.018918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791c2e8-b6e5-4ace-9e7f-c7b0427374e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:52.019520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791c2e8-b6e5-4ace-9e7f-c7b0427374e3 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:52.019799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791c2e8-b6e5-4ace-9e7f-c7b0427374e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:52.020068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791c2e8-b6e5-4ace-9e7f-c7b0427374e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:52.020790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791c2e8-b6e5-4ace-9e7f-c7b0427374e3 None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=34ec5c9c71d14762a4869b7d66e87ee3) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:52.021098 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791c2e8-b6e5-4ace-9e7f-c7b0427374e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:52.022701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791c2e8-b6e5-4ace-9e7f-c7b0427374e3 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:52.028051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 628/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:33:51 2022] GET /identity/v3/role_assignments?user.id=34ec5c9c71d14762a4869b7d66e87ee3&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:54.214547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b9f42-bd46-4e5f-8a62-de22521dd926 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:54.215080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b9f42-bd46-4e5f-8a62-de22521dd926 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:54.215080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b9f42-bd46-4e5f-8a62-de22521dd926 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:54.216531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 633/1261] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:33:54.221783 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88509b6c-110d-4f0e-9289-40d1ee31e096 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:54.222241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88509b6c-110d-4f0e-9289-40d1ee31e096 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:54.222332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88509b6c-110d-4f0e-9289-40d1ee31e096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:54.241555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-88509b6c-110d-4f0e-9289-40d1ee31e096 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:33:54.242838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-88509b6c-110d-4f0e-9289-40d1ee31e096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:33:54.307556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 629/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:33:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:33:54.577051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b9f42-bd46-4e5f-8a62-de22521dd926 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:54.587813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:54.679529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b9f42-bd46-4e5f-8a62-de22521dd926 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:54.714015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a809e8-ca3f-4ffe-a2d8-b450e175cb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:54.714645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a809e8-ca3f-4ffe-a2d8-b450e175cb72 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:54.714929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a809e8-ca3f-4ffe-a2d8-b450e175cb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:54.715205 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a809e8-ca3f-4ffe-a2d8-b450e175cb72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:54.726098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a809e8-ca3f-4ffe-a2d8-b450e175cb72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d3a809e8-ca3f-4ffe-a2d8-b450e175cb72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:33:54.730903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:54.732036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:54.733216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 634/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:33:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:54.738176 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88509b6c-110d-4f0e-9289-40d1ee31e096 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:54.756745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:54.839508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88509b6c-110d-4f0e-9289-40d1ee31e096 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:54.873522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e31856-5280-44cc-9531-efdb89fee3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:54.874140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e31856-5280-44cc-9531-efdb89fee3da None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:54.874440 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e31856-5280-44cc-9531-efdb89fee3da None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:54.874704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e31856-5280-44cc-9531-efdb89fee3da None admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:54.875353 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e31856-5280-44cc-9531-efdb89fee3da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:54.875648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e31856-5280-44cc-9531-efdb89fee3da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:54.876996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e31856-5280-44cc-9531-efdb89fee3da None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:54.884874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 630/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:33:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:54.890760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a809e8-ca3f-4ffe-a2d8-b450e175cb72 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:54.957265 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:54.992378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a809e8-ca3f-4ffe-a2d8-b450e175cb72 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:55.027272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c77ca8-c12e-4280-9f01-2e7a5daed1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:55.027893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c77ca8-c12e-4280-9f01-2e7a5daed1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:55.028172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c77ca8-c12e-4280-9f01-2e7a5daed1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:55.028455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c77ca8-c12e-4280-9f01-2e7a5daed1a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:55.040137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c77ca8-c12e-4280-9f01-2e7a5daed1a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:55.041586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c77ca8-c12e-4280-9f01-2e7a5daed1a7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-28c77ca8-c12e-4280-9f01-2e7a5daed1a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:55.055774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:55.056910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:55.064081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:55.064081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:55.064081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:55.064081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 635/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:55.064378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e31856-5280-44cc-9531-efdb89fee3da None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:55.143169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:55.161755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e31856-5280-44cc-9531-efdb89fee3da None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:55.195422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cebc55-9241-4d66-a5f3-33927517f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:55.196013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cebc55-9241-4d66-a5f3-33927517f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:55.196431 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cebc55-9241-4d66-a5f3-33927517f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:55.196703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cebc55-9241-4d66-a5f3-33927517f7c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:55.197332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cebc55-9241-4d66-a5f3-33927517f7c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:55.197628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cebc55-9241-4d66-a5f3-33927517f7c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:55.199279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cebc55-9241-4d66-a5f3-33927517f7c5 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:55.211871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 631/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:55.217766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c77ca8-c12e-4280-9f01-2e7a5daed1a7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:55.314851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c77ca8-c12e-4280-9f01-2e7a5daed1a7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:55.336899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:55.351179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d03d3b-dce3-455d-86cc-651db1fe3b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:55.351809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d03d3b-dce3-455d-86cc-651db1fe3b0d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:55.352090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d03d3b-dce3-455d-86cc-651db1fe3b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:55.352365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d03d3b-dce3-455d-86cc-651db1fe3b0d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:55.359733 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d03d3b-dce3-455d-86cc-651db1fe3b0d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:55.361162 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d03d3b-dce3-455d-86cc-651db1fe3b0d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-29d03d3b-dce3-455d-86cc-651db1fe3b0d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:33:55.372096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:55.373251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:55.374448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 636/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:55.379049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cebc55-9241-4d66-a5f3-33927517f7c5 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:55.478605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cebc55-9241-4d66-a5f3-33927517f7c5 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:55.512238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1583b-de89-41d5-96a8-2d11fb0d6b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:55.512835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1583b-de89-41d5-96a8-2d11fb0d6b49 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:55.513113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1583b-de89-41d5-96a8-2d11fb0d6b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:55.513384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1583b-de89-41d5-96a8-2d11fb0d6b49 None admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:55.514128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1583b-de89-41d5-96a8-2d11fb0d6b49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:55.514431 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1583b-de89-41d5-96a8-2d11fb0d6b49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:55.515856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1583b-de89-41d5-96a8-2d11fb0d6b49 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:55.525038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:55.533014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 632/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Sep 17 03:33:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:55.539289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d03d3b-dce3-455d-86cc-651db1fe3b0d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:55.638779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d03d3b-dce3-455d-86cc-651db1fe3b0d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:55.670689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3c406-6321-4a46-bf10-be4888f19069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:55.671295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3c406-6321-4a46-bf10-be4888f19069 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:55.671576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3c406-6321-4a46-bf10-be4888f19069 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:55.671850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3c406-6321-4a46-bf10-be4888f19069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:55.683353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3c406-6321-4a46-bf10-be4888f19069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:55.684800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3c406-6321-4a46-bf10-be4888f19069 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f8e3c406-6321-4a46-bf10-be4888f19069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:55.699334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:55.700495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:55.701618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:55.701618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:55.701618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:55.701618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 637/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:55.706129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1583b-de89-41d5-96a8-2d11fb0d6b49 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:55.808895 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1583b-de89-41d5-96a8-2d11fb0d6b49 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:55.842777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532435c1-b06a-4dde-8302-60415e27c883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:55.843378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532435c1-b06a-4dde-8302-60415e27c883 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:55.843655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532435c1-b06a-4dde-8302-60415e27c883 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:55.843924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532435c1-b06a-4dde-8302-60415e27c883 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:55.844553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532435c1-b06a-4dde-8302-60415e27c883 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:55.844896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532435c1-b06a-4dde-8302-60415e27c883 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:55.846325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532435c1-b06a-4dde-8302-60415e27c883 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:55.857540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 633/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:55.863384 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3c406-6321-4a46-bf10-be4888f19069 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:55.894986 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:55.962234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3c406-6321-4a46-bf10-be4888f19069 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:55.997011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9647ec-7a83-44fb-80e5-48ac234577e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:55.997640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9647ec-7a83-44fb-80e5-48ac234577e8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:55.997951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9647ec-7a83-44fb-80e5-48ac234577e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:55.998249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9647ec-7a83-44fb-80e5-48ac234577e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:56.004863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9647ec-7a83-44fb-80e5-48ac234577e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:56.006775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9647ec-7a83-44fb-80e5-48ac234577e8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-6e9647ec-7a83-44fb-80e5-48ac234577e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:33:56.017809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:56.024020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:56.025308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:33:56.025308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:33:56.025308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:33:56.025308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:33:56.025308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:33:56.025308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:33:56.025308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:33:56.025308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:33:56.025308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 638/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:33:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:56.025961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532435c1-b06a-4dde-8302-60415e27c883 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:56.085118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:56.124397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532435c1-b06a-4dde-8302-60415e27c883 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:56.157557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65593c79-d44f-42f8-92f2-78da41bf8b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:56.158188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65593c79-d44f-42f8-92f2-78da41bf8b58 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:56.158466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65593c79-d44f-42f8-92f2-78da41bf8b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:56.158737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65593c79-d44f-42f8-92f2-78da41bf8b58 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:56.159419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65593c79-d44f-42f8-92f2-78da41bf8b58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:56.159724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65593c79-d44f-42f8-92f2-78da41bf8b58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:56.161196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65593c79-d44f-42f8-92f2-78da41bf8b58 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:56.172830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 634/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:33:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:33:56.179143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9647ec-7a83-44fb-80e5-48ac234577e8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:56.269941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:56.278700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9647ec-7a83-44fb-80e5-48ac234577e8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:56.311867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e200e72-ca97-4633-8e80-d699da0ee360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:56.312520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e200e72-ca97-4633-8e80-d699da0ee360 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:56.312804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e200e72-ca97-4633-8e80-d699da0ee360 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:56.313084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e200e72-ca97-4633-8e80-d699da0ee360 None admin] PATH_INFO: `/v3/projects/bef114f2baf2415ba9942c8358a243db/users/34ec5c9c71d14762a4869b7d66e87ee3/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:56.314999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e200e72-ca97-4633-8e80-d699da0ee360 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef114f2baf2415ba9942c8358a243db, user_id=34ec5c9c71d14762a4869b7d66e87ee3, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:56.316455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e200e72-ca97-4633-8e80-d699da0ee360 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:56.397352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 639/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Sep 17 03:33:56 2022] PUT /identity/v3/projects/bef114f2baf2415ba9942c8358a243db/users/34ec5c9c71d14762a4869b7d66e87ee3/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:33:58.546566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16134fc-28cc-4b75-9ea3-d79e86899625 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:58.547126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16134fc-28cc-4b75-9ea3-d79e86899625 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:58.547126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16134fc-28cc-4b75-9ea3-d79e86899625 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:58.548530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 635/1274] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:33:58.554979 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f250a64-4af3-4f75-b7f4-5ba1aecafdf1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:58.555456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f250a64-4af3-4f75-b7f4-5ba1aecafdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:58.558143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f250a64-4af3-4f75-b7f4-5ba1aecafdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:58.575577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7f250a64-4af3-4f75-b7f4-5ba1aecafdf1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:33:58.576810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7f250a64-4af3-4f75-b7f4-5ba1aecafdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:33:58.646505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 640/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:33:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:33:58.916290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16134fc-28cc-4b75-9ea3-d79e86899625 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:59.019066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16134fc-28cc-4b75-9ea3-d79e86899625 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:59.035909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:59.053242 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e07dfd-0be6-4c44-8ed7-f031b021db44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:59.053902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e07dfd-0be6-4c44-8ed7-f031b021db44 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:59.054194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e07dfd-0be6-4c44-8ed7-f031b021db44 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:59.054448 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e07dfd-0be6-4c44-8ed7-f031b021db44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:59.064199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e07dfd-0be6-4c44-8ed7-f031b021db44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-a2e07dfd-0be6-4c44-8ed7-f031b021db44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 03:33:59.068935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:59.070020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 03:33:59.070984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:59.074070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 636/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Sep 17 03:33:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:59.076336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f250a64-4af3-4f75-b7f4-5ba1aecafdf1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:59.154811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:59.172704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f250a64-4af3-4f75-b7f4-5ba1aecafdf1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:59.205793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7486438-9e61-4242-a2cd-6e693ceca927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:59.206397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7486438-9e61-4242-a2cd-6e693ceca927 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:59.206675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7486438-9e61-4242-a2cd-6e693ceca927 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:59.206953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7486438-9e61-4242-a2cd-6e693ceca927 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:59.207612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7486438-9e61-4242-a2cd-6e693ceca927 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:59.207917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7486438-9e61-4242-a2cd-6e693ceca927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:59.209400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7486438-9e61-4242-a2cd-6e693ceca927 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:59.217182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 641/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Sep 17 03:33:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:59.223047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e07dfd-0be6-4c44-8ed7-f031b021db44 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:59.327434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e07dfd-0be6-4c44-8ed7-f031b021db44 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:59.361848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138c854-38d0-4c2f-b8ce-591b9f20e371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:59.362468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138c854-38d0-4c2f-b8ce-591b9f20e371 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:59.362763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138c854-38d0-4c2f-b8ce-591b9f20e371 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:59.363029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138c854-38d0-4c2f-b8ce-591b9f20e371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:59.373051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138c854-38d0-4c2f-b8ce-591b9f20e371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:59.374511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138c854-38d0-4c2f-b8ce-591b9f20e371 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-a138c854-38d0-4c2f-b8ce-591b9f20e371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:33:59.388853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:59.399857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:33:59.400786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:33:59.400786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:33:59.400786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:59.400786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 637/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:59.401022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7486438-9e61-4242-a2cd-6e693ceca927 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:59.497019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7486438-9e61-4242-a2cd-6e693ceca927 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:59.529586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bb758-e5ff-4aa6-8827-560d56402172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:59.530203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bb758-e5ff-4aa6-8827-560d56402172 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:59.530485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bb758-e5ff-4aa6-8827-560d56402172 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:59.530752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bb758-e5ff-4aa6-8827-560d56402172 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:59.531378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bb758-e5ff-4aa6-8827-560d56402172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:59.531676 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bb758-e5ff-4aa6-8827-560d56402172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:59.533023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bb758-e5ff-4aa6-8827-560d56402172 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:59.544347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 642/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:59.549992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138c854-38d0-4c2f-b8ce-591b9f20e371 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:59.581689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:59.651536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138c854-38d0-4c2f-b8ce-591b9f20e371 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:59.684554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8a5ec-8f57-4578-ba3c-a0043e8138ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:59.685175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8a5ec-8f57-4578-ba3c-a0043e8138ee None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:59.685449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8a5ec-8f57-4578-ba3c-a0043e8138ee None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:59.685739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8a5ec-8f57-4578-ba3c-a0043e8138ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:59.692930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8a5ec-8f57-4578-ba3c-a0043e8138ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:59.694406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8a5ec-8f57-4578-ba3c-a0043e8138ee None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-38a8a5ec-8f57-4578-ba3c-a0043e8138ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 03:33:59.705508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:33:59.706711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:33:59.708038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 638/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:33:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:33:59.712587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bb758-e5ff-4aa6-8827-560d56402172 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:59.716457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:59.817983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bb758-e5ff-4aa6-8827-560d56402172 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:33:59.851405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57350d-2bf5-4de1-af22-dd27dc67713d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:33:59.851950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57350d-2bf5-4de1-af22-dd27dc67713d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:33:59.852229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57350d-2bf5-4de1-af22-dd27dc67713d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:33:59.852508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57350d-2bf5-4de1-af22-dd27dc67713d None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:33:59.853255 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57350d-2bf5-4de1-af22-dd27dc67713d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:33:59.853564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57350d-2bf5-4de1-af22-dd27dc67713d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:33:59.855059 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57350d-2bf5-4de1-af22-dd27dc67713d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:33:59.875552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 643/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Sep 17 03:33:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:33:59.881515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8a5ec-8f57-4578-ba3c-a0043e8138ee None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:33:59.979494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:33:59.989228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8a5ec-8f57-4578-ba3c-a0043e8138ee None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:00.026142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008993b-d6ca-481c-af24-33e92cd5cf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:00.026750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008993b-d6ca-481c-af24-33e92cd5cf13 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:00.027025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008993b-d6ca-481c-af24-33e92cd5cf13 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:00.027293 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008993b-d6ca-481c-af24-33e92cd5cf13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:00.101903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008993b-d6ca-481c-af24-33e92cd5cf13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:00.103309 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008993b-d6ca-481c-af24-33e92cd5cf13 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:00.202811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:00.214627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 639/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Sep 17 03:33:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 03:34:00.220497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57350d-2bf5-4de1-af22-dd27dc67713d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:00.284559 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:00.325792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57350d-2bf5-4de1-af22-dd27dc67713d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:00.358380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a29dc0-746f-4872-9cb5-01e2336303d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:00.358935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29dc0-746f-4872-9cb5-01e2336303d9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:00.359211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29dc0-746f-4872-9cb5-01e2336303d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:00.359496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29dc0-746f-4872-9cb5-01e2336303d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:00.370940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a29dc0-746f-4872-9cb5-01e2336303d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:00.372408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a29dc0-746f-4872-9cb5-01e2336303d9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e6a29dc0-746f-4872-9cb5-01e2336303d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 03:34:00.386767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:34:00.387916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 03:34:00.389077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 03:34:00.389077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 03:34:00.389077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:34:00.389077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 644/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:34:00.393694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008993b-d6ca-481c-af24-33e92cd5cf13 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:00.493236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008993b-d6ca-481c-af24-33e92cd5cf13 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:00.527889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3c454-25bb-4acc-ac7a-4c4c42768534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:00.528496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3c454-25bb-4acc-ac7a-4c4c42768534 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:00.528768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3c454-25bb-4acc-ac7a-4c4c42768534 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:00.529034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3c454-25bb-4acc-ac7a-4c4c42768534 None admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:00.529662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3c454-25bb-4acc-ac7a-4c4c42768534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:34:00.529988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3c454-25bb-4acc-ac7a-4c4c42768534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:00.531351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3c454-25bb-4acc-ac7a-4c4c42768534 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:00.543322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 640/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Sep 17 03:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:34:00.549456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a29dc0-746f-4872-9cb5-01e2336303d9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:00.655329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a29dc0-746f-4872-9cb5-01e2336303d9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:00.669707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:00.689863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1a4a3-5e25-45a0-961a-aa80117c3a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:00.690481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1a4a3-5e25-45a0-961a-aa80117c3a07 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:00.690759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1a4a3-5e25-45a0-961a-aa80117c3a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:00.691031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1a4a3-5e25-45a0-961a-aa80117c3a07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:00.698135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1a4a3-5e25-45a0-961a-aa80117c3a07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:00.699824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1a4a3-5e25-45a0-961a-aa80117c3a07 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-0cd1a4a3-5e25-45a0-961a-aa80117c3a07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 03:34:00.711467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:34:00.712538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:34:00.714214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 03:34:00.714214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 03:34:00.714214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 03:34:00.714214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 03:34:00.714214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 03:34:00.714214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 03:34:00.714214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 03:34:00.714214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Sep 17 03:34:00.714214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 645/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Sep 17 03:34:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:34:00.718528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3c454-25bb-4acc-ac7a-4c4c42768534 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:00.768383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:00.818880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3c454-25bb-4acc-ac7a-4c4c42768534 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:00.850826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d12779-e1fc-4902-abf9-b6bf682c4600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:00.851424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d12779-e1fc-4902-abf9-b6bf682c4600 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:00.851702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d12779-e1fc-4902-abf9-b6bf682c4600 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:00.851974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d12779-e1fc-4902-abf9-b6bf682c4600 None admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:00.852681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d12779-e1fc-4902-abf9-b6bf682c4600 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:34:00.852987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d12779-e1fc-4902-abf9-b6bf682c4600 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:00.854448 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d12779-e1fc-4902-abf9-b6bf682c4600 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:00.866517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 641/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Sep 17 03:34:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:34:00.872894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1a4a3-5e25-45a0-961a-aa80117c3a07 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:00.973222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1a4a3-5e25-45a0-961a-aa80117c3a07 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:01.006445 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df96c68-8fa4-486b-a056-8561b4d0665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:01.009308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df96c68-8fa4-486b-a056-8561b4d0665e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:01.009308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df96c68-8fa4-486b-a056-8561b4d0665e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:01.009308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df96c68-8fa4-486b-a056-8561b4d0665e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:01.009308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df96c68-8fa4-486b-a056-8561b4d0665e None admin] RBAC: Adding query filter params (role.id=33ff6e45fa82455ab009b3b37a581e49, scope.project.id=bef114f2baf2415ba9942c8358a243db, user.id=34ec5c9c71d14762a4869b7d66e87ee3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:34:01.009308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df96c68-8fa4-486b-a056-8561b4d0665e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:01.010047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df96c68-8fa4-486b-a056-8561b4d0665e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:01.015744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 646/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Sep 17 03:34:00 2022] GET /identity/v3/role_assignments?user.id=34ec5c9c71d14762a4869b7d66e87ee3&scope.project.id=bef114f2baf2415ba9942c8358a243db&role.id=33ff6e45fa82455ab009b3b37a581e49 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:34:03.204948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d7e8c-18c3-4958-80a5-fe6c9564b301 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:03.205498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d7e8c-18c3-4958-80a5-fe6c9564b301 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:03.205498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d7e8c-18c3-4958-80a5-fe6c9564b301 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:03.206876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 642/1288] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:34:03.212765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56806f9e-9e62-4fbf-ab79-e5e7e766c15a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:03.213163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56806f9e-9e62-4fbf-ab79-e5e7e766c15a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:03.213324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56806f9e-9e62-4fbf-ab79-e5e7e766c15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:03.232290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-56806f9e-9e62-4fbf-ab79-e5e7e766c15a None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:34:03.234442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-56806f9e-9e62-4fbf-ab79-e5e7e766c15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:34:03.262618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:03.299340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 647/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:34:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:34:03.578394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d7e8c-18c3-4958-80a5-fe6c9564b301 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:03.677918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d7e8c-18c3-4958-80a5-fe6c9564b301 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:03.688885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:03.711883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8578bb0-23df-4435-8219-ca8c6c1d7b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:03.712498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8578bb0-23df-4435-8219-ca8c6c1d7b8f None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:03.712775 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8578bb0-23df-4435-8219-ca8c6c1d7b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:03.713045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8578bb0-23df-4435-8219-ca8c6c1d7b8f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:03.713617 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8578bb0-23df-4435-8219-ca8c6c1d7b8f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:03.715016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8578bb0-23df-4435-8219-ca8c6c1d7b8f None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-d8578bb0-23df-4435-8219-ca8c6c1d7b8f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:34:03.723467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:34:03.724670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:34:03.726260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 643/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Sep 17 03:34:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:34:03.731243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56806f9e-9e62-4fbf-ab79-e5e7e766c15a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:03.828163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56806f9e-9e62-4fbf-ab79-e5e7e766c15a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:03.861764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d6648-c6fc-47dc-b5e8-274faf018ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:03.862307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d6648-c6fc-47dc-b5e8-274faf018ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:03.862586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d6648-c6fc-47dc-b5e8-274faf018ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:03.862858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d6648-c6fc-47dc-b5e8-274faf018ad6 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:03.863482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d6648-c6fc-47dc-b5e8-274faf018ad6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:34:03.863780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d6648-c6fc-47dc-b5e8-274faf018ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:03.865172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d6648-c6fc-47dc-b5e8-274faf018ad6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:03.869935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 648/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Sat Sep 17 03:34:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:34:03.875876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8578bb0-23df-4435-8219-ca8c6c1d7b8f None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:03.973965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8578bb0-23df-4435-8219-ca8c6c1d7b8f None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:04.013093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910e4e0-168c-4cf7-a827-2c87b5f2563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:04.013706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910e4e0-168c-4cf7-a827-2c87b5f2563e None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:04.014009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910e4e0-168c-4cf7-a827-2c87b5f2563e None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:04.014275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910e4e0-168c-4cf7-a827-2c87b5f2563e None admin] PATH_INFO: `/v3/services` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:04.014887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910e4e0-168c-4cf7-a827-2c87b5f2563e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:34:04.015181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910e4e0-168c-4cf7-a827-2c87b5f2563e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:04.016581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910e4e0-168c-4cf7-a827-2c87b5f2563e None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:04.020962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 644/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Sat Sep 17 03:34:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:34:06.145101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08089986-2148-427c-aab8-be5e210735ac None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:06.145609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08089986-2148-427c-aab8-be5e210735ac None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:06.145609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08089986-2148-427c-aab8-be5e210735ac None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:06.151052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 649/1293] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:34:06.152922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6fbf-9c29-459d-b5e5-954a162ec655 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:06.153354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6fbf-9c29-459d-b5e5-954a162ec655 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:06.153501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6fbf-9c29-459d-b5e5-954a162ec655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:06.172071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-f72d6fbf-9c29-459d-b5e5-954a162ec655 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:34:06.173257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f72d6fbf-9c29-459d-b5e5-954a162ec655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:34:06.202501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:06.241335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 645/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:34:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:34:06.528863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08089986-2148-427c-aab8-be5e210735ac None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:06.561867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:06.641669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08089986-2148-427c-aab8-be5e210735ac None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:06.675207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca778440-921d-4203-afaa-1838a7c3efc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:06.675794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca778440-921d-4203-afaa-1838a7c3efc5 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:06.676075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca778440-921d-4203-afaa-1838a7c3efc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:06.676349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca778440-921d-4203-afaa-1838a7c3efc5 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:06.676995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca778440-921d-4203-afaa-1838a7c3efc5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:06.678547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca778440-921d-4203-afaa-1838a7c3efc5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:06.776872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 650/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Sep 17 03:34:06 2022] POST /identity/v3/services => generated 246 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:34:09.034973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22412e5-0583-4e44-91ab-0dfb6e98fdac None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:09.035524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22412e5-0583-4e44-91ab-0dfb6e98fdac None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:09.035524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22412e5-0583-4e44-91ab-0dfb6e98fdac None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:09.036858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 646/1296] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:34:09.042711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff07568-6f8e-4de7-9213-3a8742d646d2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:09.043118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff07568-6f8e-4de7-9213-3a8742d646d2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:09.043267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff07568-6f8e-4de7-9213-3a8742d646d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:09.063378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9ff07568-6f8e-4de7-9213-3a8742d646d2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:34:09.063378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9ff07568-6f8e-4de7-9213-3a8742d646d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:34:09.134901 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 651/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:34:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:34:09.403116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22412e5-0583-4e44-91ab-0dfb6e98fdac None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:09.501920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22412e5-0583-4e44-91ab-0dfb6e98fdac None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:09.536051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418814c6-3045-437c-bccd-e5a1e93a6bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:09.536668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418814c6-3045-437c-bccd-e5a1e93a6bde None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:09.536944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418814c6-3045-437c-bccd-e5a1e93a6bde None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:09.537216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418814c6-3045-437c-bccd-e5a1e93a6bde None admin] PATH_INFO: `/v3/services/placement` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:09.538124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418814c6-3045-437c-bccd-e5a1e93a6bde None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:09.539488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418814c6-3045-437c-bccd-e5a1e93a6bde None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-418814c6-3045-437c-bccd-e5a1e93a6bde None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:34:09.548047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:34:09.549210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:34:09.550322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 647/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Sep 17 03:34:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:34:09.555295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff07568-6f8e-4de7-9213-3a8742d646d2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:09.573582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:09.651983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff07568-6f8e-4de7-9213-3a8742d646d2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:09.684343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9156ca-b3d6-4909-a4ba-b3163f06916a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:09.684861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9156ca-b3d6-4909-a4ba-b3163f06916a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:09.685139 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9156ca-b3d6-4909-a4ba-b3163f06916a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:09.685409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9156ca-b3d6-4909-a4ba-b3163f06916a None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:09.686057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9156ca-b3d6-4909-a4ba-b3163f06916a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:34:09.686354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9156ca-b3d6-4909-a4ba-b3163f06916a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:09.687707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9156ca-b3d6-4909-a4ba-b3163f06916a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:09.692642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 652/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Sat Sep 17 03:34:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:34:09.699530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418814c6-3045-437c-bccd-e5a1e93a6bde None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:09.731075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:09.799310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418814c6-3045-437c-bccd-e5a1e93a6bde None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:09.833878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800f015-bc63-42a8-b312-d49d77b55caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:09.834477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800f015-bc63-42a8-b312-d49d77b55caa None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:09.834752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800f015-bc63-42a8-b312-d49d77b55caa None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:09.835020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800f015-bc63-42a8-b312-d49d77b55caa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:09.835708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800f015-bc63-42a8-b312-d49d77b55caa None admin] RBAC: Adding query filter params (interface=public, service_id=7a4e4e2baf944edbae443857589b6773, region_id=RegionOne) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:34:09.836011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800f015-bc63-42a8-b312-d49d77b55caa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:09.837366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800f015-bc63-42a8-b312-d49d77b55caa None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:09.842330 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 648/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Sep 17 03:34:09 2022] GET /identity/v3/endpoints?service_id=7a4e4e2baf944edbae443857589b6773&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:34:09.848085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9156ca-b3d6-4909-a4ba-b3163f06916a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:09.932149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:09.950217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9156ca-b3d6-4909-a4ba-b3163f06916a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:09.982624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5e735-81a0-46e3-ad63-b55f8b1f3abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:09.983218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5e735-81a0-46e3-ad63-b55f8b1f3abb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:09.983499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5e735-81a0-46e3-ad63-b55f8b1f3abb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:09.983775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5e735-81a0-46e3-ad63-b55f8b1f3abb None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:09.984367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5e735-81a0-46e3-ad63-b55f8b1f3abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:09.985754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5e735-81a0-46e3-ad63-b55f8b1f3abb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:09.993168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 653/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Sep 17 03:34:09 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:34:12.194857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380ad29-320f-40ba-a228-07d98faf4732 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:12.195482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380ad29-320f-40ba-a228-07d98faf4732 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:12.195769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380ad29-320f-40ba-a228-07d98faf4732 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:12.197166 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 649/1302] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:34:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:34:12.202782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8331-5770-4c69-a9c0-3fe7cbaa8234 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:12.203201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8331-5770-4c69-a9c0-3fe7cbaa8234 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:12.203370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8331-5770-4c69-a9c0-3fe7cbaa8234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:12.221376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5f3d8331-5770-4c69-a9c0-3fe7cbaa8234 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:34:12.222577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5f3d8331-5770-4c69-a9c0-3fe7cbaa8234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:34:12.290081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 654/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:34:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 03:34:12.558336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380ad29-320f-40ba-a228-07d98faf4732 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:12.662032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:12.668959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380ad29-320f-40ba-a228-07d98faf4732 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:12.701862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c054ae1-0e5f-4863-adcd-1fb304d9a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:12.702490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c054ae1-0e5f-4863-adcd-1fb304d9a9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:12.702771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c054ae1-0e5f-4863-adcd-1fb304d9a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:12.703040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c054ae1-0e5f-4863-adcd-1fb304d9a9d4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:12.703613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c054ae1-0e5f-4863-adcd-1fb304d9a9d4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:12.704976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c054ae1-0e5f-4863-adcd-1fb304d9a9d4 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application [None req-3c054ae1-0e5f-4863-adcd-1fb304d9a9d4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 03:34:12.714031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 03:34:12.715208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application with Lock( Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self._enter() Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application return self.creator() Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application created_value = creator( Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: ERROR keystone.server.flask.application Sep 17 03:34:12.716122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 650/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Sep 17 03:34:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 03:34:12.721211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d8331-5770-4c69-a9c0-3fe7cbaa8234 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:12.797166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:12.820990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d8331-5770-4c69-a9c0-3fe7cbaa8234 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:12.855656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7ba01-d56a-4b43-aaa6-1635ad0f2d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:12.856251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7ba01-d56a-4b43-aaa6-1635ad0f2d4a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:12.856529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7ba01-d56a-4b43-aaa6-1635ad0f2d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:12.856828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7ba01-d56a-4b43-aaa6-1635ad0f2d4a None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:12.857449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7ba01-d56a-4b43-aaa6-1635ad0f2d4a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:34:12.857770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7ba01-d56a-4b43-aaa6-1635ad0f2d4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:12.859123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7ba01-d56a-4b43-aaa6-1635ad0f2d4a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:12.864037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 655/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Sat Sep 17 03:34:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:34:12.870222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c054ae1-0e5f-4863-adcd-1fb304d9a9d4 None admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:34:12.971822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c054ae1-0e5f-4863-adcd-1fb304d9a9d4 None admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:34:13.004429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e60dfe-ba84-46a2-b0ba-c6d735316b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:34:13.005005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e60dfe-ba84-46a2-b0ba-c6d735316b84 None admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:34:13.005277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e60dfe-ba84-46a2-b0ba-c6d735316b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:34:13.005633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e60dfe-ba84-46a2-b0ba-c6d735316b84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:34:13.006595 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e60dfe-ba84-46a2-b0ba-c6d735316b84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:34:13.007757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e60dfe-ba84-46a2-b0ba-c6d735316b84 None admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:34:13.086148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 651/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:34:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:34:22.861094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=30 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:34:23.080928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:10.998590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854274dc-068c-4ddb-aa45-b750dc8ea871 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:10.999339 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854274dc-068c-4ddb-aa45-b750dc8ea871 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:10.999339 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854274dc-068c-4ddb-aa45-b750dc8ea871 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:11.006251 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d3575-79fb-4c6c-8e0e-cf0d14d3a907 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:11.006952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d3575-79fb-4c6c-8e0e-cf0d14d3a907 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:11.006952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d3575-79fb-4c6c-8e0e-cf0d14d3a907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:11.009521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 656/1307] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:11.088782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-656d3575-79fb-4c6c-8e0e-cf0d14d3a907 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:11.088782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-656d3575-79fb-4c6c-8e0e-cf0d14d3a907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:11.206244 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 652/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:11.492515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dddacc-f768-4937-a0ed-8efc7e41c48b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:11.492798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dddacc-f768-4937-a0ed-8efc7e41c48b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:11.493237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dddacc-f768-4937-a0ed-8efc7e41c48b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:11.494638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 657/1309] 10.222.0.42 () {54 vars in 963 bytes} [Sat Sep 17 03:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:11.501804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dca645-6205-462d-bd1c-1c82752fb180 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:11.502082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dca645-6205-462d-bd1c-1c82752fb180 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:11.502554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dca645-6205-462d-bd1c-1c82752fb180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:11.590667 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b7dca645-6205-462d-bd1c-1c82752fb180 None None] MFA Rules not processed for user `6336f0c469b84db78a90cf8435c7dbe4`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:11.592447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b7dca645-6205-462d-bd1c-1c82752fb180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:11.618236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:11.681944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 653/1310] 10.222.0.42 () {58 vars in 1060 bytes} [Sat Sep 17 03:40:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:11.691915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dddacc-f768-4937-a0ed-8efc7e41c48b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:11.693634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c9dddacc-f768-4937-a0ed-8efc7e41c48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:11.822885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dddacc-f768-4937-a0ed-8efc7e41c48b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:11.912812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688acb8-2e0b-493e-ac03-3d8612326d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:11.913549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688acb8-2e0b-493e-ac03-3d8612326d10 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:11.913856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688acb8-2e0b-493e-ac03-3d8612326d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:11.914157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688acb8-2e0b-493e-ac03-3d8612326d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:11.915811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6688acb8-2e0b-493e-ac03-3d8612326d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:11.980191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:12.035698 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688acb8-2e0b-493e-ac03-3d8612326d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:12.037261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688acb8-2e0b-493e-ac03-3d8612326d10 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:12.057405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 658/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:14.554519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad056c72-7e06-4e64-ad04-62244d9afe0f None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:14.555047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad056c72-7e06-4e64-ad04-62244d9afe0f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:14.555047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad056c72-7e06-4e64-ad04-62244d9afe0f None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:14.556325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 654/1312] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:14.564548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182f275-601e-4ed2-a4ca-4abac9802c6f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:14.565365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182f275-601e-4ed2-a4ca-4abac9802c6f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:14.565365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182f275-601e-4ed2-a4ca-4abac9802c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:14.654362 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a182f275-601e-4ed2-a4ca-4abac9802c6f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:14.655563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a182f275-601e-4ed2-a4ca-4abac9802c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:14.749039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 659/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:15.028365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad056c72-7e06-4e64-ad04-62244d9afe0f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:15.030076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ad056c72-7e06-4e64-ad04-62244d9afe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:15.132883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad056c72-7e06-4e64-ad04-62244d9afe0f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:15.208505 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:15.210961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73041250-7b6b-4ff3-81ca-578873687b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:15.211569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73041250-7b6b-4ff3-81ca-578873687b6e service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:15.211841 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73041250-7b6b-4ff3-81ca-578873687b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:15.212106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73041250-7b6b-4ff3-81ca-578873687b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:15.213695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-73041250-7b6b-4ff3-81ca-578873687b6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:15.323518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73041250-7b6b-4ff3-81ca-578873687b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:15.324858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73041250-7b6b-4ff3-81ca-578873687b6e service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:15.344683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 655/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:18.005591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4089009-a015-42fb-aad1-4598f0ef5dbc None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:18.006155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4089009-a015-42fb-aad1-4598f0ef5dbc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:18.006155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4089009-a015-42fb-aad1-4598f0ef5dbc None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:18.007511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 660/1315] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:18.012821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efd4e3-e5cb-483f-a2dd-3d2fb4671597 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:18.013255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efd4e3-e5cb-483f-a2dd-3d2fb4671597 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:18.013347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efd4e3-e5cb-483f-a2dd-3d2fb4671597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:18.078785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-43efd4e3-e5cb-483f-a2dd-3d2fb4671597 None None] MFA Rules not processed for user `c3e17b4e2c274478a5af0c474f6198f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:18.079903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-43efd4e3-e5cb-483f-a2dd-3d2fb4671597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:18.130785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:18.175367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 656/1316] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:18.475792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c001a-47e6-4c40-bf06-efa8f9ee7a9d None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:18.476076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c001a-47e6-4c40-bf06-efa8f9ee7a9d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:18.476357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c001a-47e6-4c40-bf06-efa8f9ee7a9d None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:18.477761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 661/1317] 10.222.0.42 () {54 vars in 963 bytes} [Sat Sep 17 03:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:18.484559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549c36c-8c7f-4a5b-ab3d-0df5987a4bd0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:18.484840 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549c36c-8c7f-4a5b-ab3d-0df5987a4bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:18.485112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549c36c-8c7f-4a5b-ab3d-0df5987a4bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:18.559715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b549c36c-8c7f-4a5b-ab3d-0df5987a4bd0 None None] MFA Rules not processed for user `6336f0c469b84db78a90cf8435c7dbe4`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:18.560899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b549c36c-8c7f-4a5b-ab3d-0df5987a4bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:18.640570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 657/1318] 10.222.0.42 () {58 vars in 1060 bytes} [Sat Sep 17 03:40:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:40:18.649635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c001a-47e6-4c40-bf06-efa8f9ee7a9d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:18.651182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bf9c001a-47e6-4c40-bf06-efa8f9ee7a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:18.662315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:18.756170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c001a-47e6-4c40-bf06-efa8f9ee7a9d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:18.831250 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6aef29-5780-4f18-a68d-bc3876354f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:18.831857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aef29-5780-4f18-a68d-bc3876354f07 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:18.832141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aef29-5780-4f18-a68d-bc3876354f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:18.832417 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aef29-5780-4f18-a68d-bc3876354f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:18.834293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8f6aef29-5780-4f18-a68d-bc3876354f07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:18.896241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:18.950459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6aef29-5780-4f18-a68d-bc3876354f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:18.951910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6aef29-5780-4f18-a68d-bc3876354f07 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:18.970021 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 662/1319] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:22.242240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995a0b1-c469-49d3-9df9-7621b57ab2a2 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:22.242802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995a0b1-c469-49d3-9df9-7621b57ab2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:22.242802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995a0b1-c469-49d3-9df9-7621b57ab2a2 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:22.244134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 658/1320] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:22.249929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16452e37-3587-4bae-807e-a7c53b3d7f30 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:22.250359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16452e37-3587-4bae-807e-a7c53b3d7f30 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:22.250478 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16452e37-3587-4bae-807e-a7c53b3d7f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:22.314975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-16452e37-3587-4bae-807e-a7c53b3d7f30 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:22.316170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-16452e37-3587-4bae-807e-a7c53b3d7f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:22.410161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 663/1321] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:22.699271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995a0b1-c469-49d3-9df9-7621b57ab2a2 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:22.701298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8995a0b1-c469-49d3-9df9-7621b57ab2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:22.751835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:22.804592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995a0b1-c469-49d3-9df9-7621b57ab2a2 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:22.878731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d7adb-9699-4590-8f6c-a985a7ba5735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:22.879236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d7adb-9699-4590-8f6c-a985a7ba5735 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:22.879510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d7adb-9699-4590-8f6c-a985a7ba5735 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:22.879779 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d7adb-9699-4590-8f6c-a985a7ba5735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:22.881365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-018d7adb-9699-4590-8f6c-a985a7ba5735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:22.976774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:22.990202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d7adb-9699-4590-8f6c-a985a7ba5735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:22.991623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d7adb-9699-4590-8f6c-a985a7ba5735 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:23.009980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 659/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:26.148605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb96cd-a4a2-4c94-b61c-8527b800ac0e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:26.149106 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb96cd-a4a2-4c94-b61c-8527b800ac0e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:26.149106 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb96cd-a4a2-4c94-b61c-8527b800ac0e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:26.150611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 664/1323] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:26.156626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fff99-d981-4c11-b15d-30ce4d6fab01 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:26.157032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fff99-d981-4c11-b15d-30ce4d6fab01 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:26.157208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fff99-d981-4c11-b15d-30ce4d6fab01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:26.223388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d19fff99-d981-4c11-b15d-30ce4d6fab01 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:26.224599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d19fff99-d981-4c11-b15d-30ce4d6fab01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:26.331560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 660/1324] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:26.619889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb96cd-a4a2-4c94-b61c-8527b800ac0e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:26.621470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-52fb96cd-a4a2-4c94-b61c-8527b800ac0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:26.641601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:26.726448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb96cd-a4a2-4c94-b61c-8527b800ac0e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:26.804645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f68ccb-28f5-4997-b19e-4196ed247004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:26.805259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68ccb-28f5-4997-b19e-4196ed247004 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:26.805540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68ccb-28f5-4997-b19e-4196ed247004 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:26.805847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68ccb-28f5-4997-b19e-4196ed247004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:26.807436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c2f68ccb-28f5-4997-b19e-4196ed247004 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:26.879410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:26.918206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f68ccb-28f5-4997-b19e-4196ed247004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:26.919945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f68ccb-28f5-4997-b19e-4196ed247004 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:26.938857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 665/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:29.176887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6f446-7aa1-47d4-bc46-9c6b16c11d09 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:29.177422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6f446-7aa1-47d4-bc46-9c6b16c11d09 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:29.177422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6f446-7aa1-47d4-bc46-9c6b16c11d09 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:29.178815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 661/1326] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:29.184598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03b68a-dac4-40d9-962e-499ac282e0c1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:29.185047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03b68a-dac4-40d9-962e-499ac282e0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:29.185386 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03b68a-dac4-40d9-962e-499ac282e0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:29.253303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9a03b68a-dac4-40d9-962e-499ac282e0c1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:29.254529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9a03b68a-dac4-40d9-962e-499ac282e0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:29.359635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 666/1327] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:29.644489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6f446-7aa1-47d4-bc46-9c6b16c11d09 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:29.646125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-acd6f446-7aa1-47d4-bc46-9c6b16c11d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:29.710916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:29.755745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6f446-7aa1-47d4-bc46-9c6b16c11d09 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:29.833186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e20bb9-521f-4e07-a371-a9c00c7c45ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:29.833818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e20bb9-521f-4e07-a371-a9c00c7c45ea service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:29.834103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e20bb9-521f-4e07-a371-a9c00c7c45ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:29.834375 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e20bb9-521f-4e07-a371-a9c00c7c45ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:29.835952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d0e20bb9-521f-4e07-a371-a9c00c7c45ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:29.951458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:29.953756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e20bb9-521f-4e07-a371-a9c00c7c45ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:29.955208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e20bb9-521f-4e07-a371-a9c00c7c45ea service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:29.975018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 662/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:32.200560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc964-eda1-4b37-9400-5e1bc5bb8cb9 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:32.201049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc964-eda1-4b37-9400-5e1bc5bb8cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:32.201141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc964-eda1-4b37-9400-5e1bc5bb8cb9 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:32.202479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 667/1329] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:32.208173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c09394-3616-4cab-844b-fe9936b6718c None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:32.208579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c09394-3616-4cab-844b-fe9936b6718c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:32.208717 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c09394-3616-4cab-844b-fe9936b6718c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:32.276007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-36c09394-3616-4cab-844b-fe9936b6718c None None] MFA Rules not processed for user `c3e17b4e2c274478a5af0c474f6198f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:32.277199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-36c09394-3616-4cab-844b-fe9936b6718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:32.374548 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 663/1330] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:32 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:32.663257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddc964-eda1-4b37-9400-5e1bc5bb8cb9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:32.664866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-80ddc964-eda1-4b37-9400-5e1bc5bb8cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:32.705660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:32.787570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddc964-eda1-4b37-9400-5e1bc5bb8cb9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:32.874614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6385e33-9057-4912-866e-61113d58cbaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:32.875145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385e33-9057-4912-866e-61113d58cbaf service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:32.875429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385e33-9057-4912-866e-61113d58cbaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:32.875704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385e33-9057-4912-866e-61113d58cbaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:32.877312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f6385e33-9057-4912-866e-61113d58cbaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:32.964635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:33.007395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6385e33-9057-4912-866e-61113d58cbaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:33.008934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6385e33-9057-4912-866e-61113d58cbaf service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:33.031858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 668/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:35.616452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452e0e9-fe1a-42b8-840e-ac78d67edc24 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:35.617011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452e0e9-fe1a-42b8-840e-ac78d67edc24 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:35.617129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452e0e9-fe1a-42b8-840e-ac78d67edc24 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:35.618492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 664/1332] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:35.624392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6b0c5-686e-46de-9205-0ed5cb295301 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:35.624771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6b0c5-686e-46de-9205-0ed5cb295301 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:35.624948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6b0c5-686e-46de-9205-0ed5cb295301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:35.690084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f8d6b0c5-686e-46de-9205-0ed5cb295301 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:35.691269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f8d6b0c5-686e-46de-9205-0ed5cb295301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:35.786120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 669/1333] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:36.064349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452e0e9-fe1a-42b8-840e-ac78d67edc24 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:36.065898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5452e0e9-fe1a-42b8-840e-ac78d67edc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:36.134151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:36.166932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452e0e9-fe1a-42b8-840e-ac78d67edc24 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:36.241227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5aa4d0-5f43-4107-a631-6c9a3be45061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:36.241856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aa4d0-5f43-4107-a631-6c9a3be45061 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:36.242134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aa4d0-5f43-4107-a631-6c9a3be45061 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:36.242402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aa4d0-5f43-4107-a631-6c9a3be45061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:36.243975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7b5aa4d0-5f43-4107-a631-6c9a3be45061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:36.350604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5aa4d0-5f43-4107-a631-6c9a3be45061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:36.351965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5aa4d0-5f43-4107-a631-6c9a3be45061 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:36.362427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:36.370481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 665/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:38.583431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac1368-df43-47fd-ade4-4e1eb7883482 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:38.584086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac1368-df43-47fd-ade4-4e1eb7883482 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:38.584086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac1368-df43-47fd-ade4-4e1eb7883482 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:38.585316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 670/1335] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:38.591290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3f65c-97a3-4411-bf76-66d4eb7c2996 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:38.591757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3f65c-97a3-4411-bf76-66d4eb7c2996 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:38.591872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3f65c-97a3-4411-bf76-66d4eb7c2996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:38.658027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4fb3f65c-97a3-4411-bf76-66d4eb7c2996 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:38.659230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4fb3f65c-97a3-4411-bf76-66d4eb7c2996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:38.754131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 666/1336] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:39.042676 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac1368-df43-47fd-ade4-4e1eb7883482 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:39.044952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-76ac1368-df43-47fd-ade4-4e1eb7883482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:39.085034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:39.148127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac1368-df43-47fd-ade4-4e1eb7883482 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:39.224201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e0859-6a02-44dc-b5ae-a14a3a8cd6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:39.224801 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e0859-6a02-44dc-b5ae-a14a3a8cd6de service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:39.225079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e0859-6a02-44dc-b5ae-a14a3a8cd6de service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:39.225349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e0859-6a02-44dc-b5ae-a14a3a8cd6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:39.226969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0f9e0859-6a02-44dc-b5ae-a14a3a8cd6de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:39.310207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:39.335544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e0859-6a02-44dc-b5ae-a14a3a8cd6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:39.336956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e0859-6a02-44dc-b5ae-a14a3a8cd6de service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:39.356494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 671/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:42.081798 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c534f-f592-4a24-80b8-fcff35f67cfd None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:42.082319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c534f-f592-4a24-80b8-fcff35f67cfd None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:42.082319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c534f-f592-4a24-80b8-fcff35f67cfd None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:42.083901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 667/1338] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:42.089073 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1835d-b5ea-4cc4-ac35-8b6ce39dfe9c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:42.089495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1835d-b5ea-4cc4-ac35-8b6ce39dfe9c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:42.089627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1835d-b5ea-4cc4-ac35-8b6ce39dfe9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:42.156853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c3e1835d-b5ea-4cc4-ac35-8b6ce39dfe9c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:42.158088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c3e1835d-b5ea-4cc4-ac35-8b6ce39dfe9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:42.254205 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 672/1339] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:42.534330 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c534f-f592-4a24-80b8-fcff35f67cfd None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:42.535843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a30c534f-f592-4a24-80b8-fcff35f67cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:42.617948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:42.645221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c534f-f592-4a24-80b8-fcff35f67cfd None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:42.722108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9d084-5056-4dd1-a72a-89eba6562999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:42.722714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9d084-5056-4dd1-a72a-89eba6562999 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:42.722990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9d084-5056-4dd1-a72a-89eba6562999 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:42.723258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9d084-5056-4dd1-a72a-89eba6562999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:42.724832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-3fb9d084-5056-4dd1-a72a-89eba6562999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:42.831673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9d084-5056-4dd1-a72a-89eba6562999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:42.833365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9d084-5056-4dd1-a72a-89eba6562999 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:42.851751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 668/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:47.276416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ad43-6dd0-4f85-a6a4-4b5d9c9e8cb1 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:47.277290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ad43-6dd0-4f85-a6a4-4b5d9c9e8cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:47.278291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ad43-6dd0-4f85-a6a4-4b5d9c9e8cb1 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:47.280289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 673/1341] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:47.286589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1cf55-3629-44ba-8b77-8b9a0af94f6f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:47.287060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1cf55-3629-44ba-8b77-8b9a0af94f6f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:47.287391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1cf55-3629-44ba-8b77-8b9a0af94f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:47.294318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:47.365988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-1be1cf55-3629-44ba-8b77-8b9a0af94f6f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:47.367185 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1be1cf55-3629-44ba-8b77-8b9a0af94f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:47.465703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 669/1342] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:47.748040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32ad43-6dd0-4f85-a6a4-4b5d9c9e8cb1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:47.749596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ec32ad43-6dd0-4f85-a6a4-4b5d9c9e8cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:47.803853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:47.862439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32ad43-6dd0-4f85-a6a4-4b5d9c9e8cb1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:47.938132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a46426-dd36-4f0f-b645-25f6f75e1630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:47.938750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46426-dd36-4f0f-b645-25f6f75e1630 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:47.939035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46426-dd36-4f0f-b645-25f6f75e1630 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:47.939306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46426-dd36-4f0f-b645-25f6f75e1630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:47.940891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a3a46426-dd36-4f0f-b645-25f6f75e1630 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:48.051745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:48.068081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a46426-dd36-4f0f-b645-25f6f75e1630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:48.069569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a46426-dd36-4f0f-b645-25f6f75e1630 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:48.088758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 674/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:51.223901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f81a3f-ada5-403c-900c-70cb4afa4194 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:51.224498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f81a3f-ada5-403c-900c-70cb4afa4194 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:51.224498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f81a3f-ada5-403c-900c-70cb4afa4194 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:51.225814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 670/1344] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:51.231671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb56d0-a93a-408a-b40a-7295e7e2c37f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:51.232090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb56d0-a93a-408a-b40a-7295e7e2c37f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:51.232269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb56d0-a93a-408a-b40a-7295e7e2c37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:51.299187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a5bb56d0-a93a-408a-b40a-7295e7e2c37f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:51.300408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a5bb56d0-a93a-408a-b40a-7295e7e2c37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:51.403587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 675/1345] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:51.691258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f81a3f-ada5-403c-900c-70cb4afa4194 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:51.692774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b6f81a3f-ada5-403c-900c-70cb4afa4194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:51.788123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:51.805942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f81a3f-ada5-403c-900c-70cb4afa4194 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:51.891807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3657b-ef85-4565-b335-5818c49947f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:51.892361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3657b-ef85-4565-b335-5818c49947f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:51.892784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3657b-ef85-4565-b335-5818c49947f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:51.893124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3657b-ef85-4565-b335-5818c49947f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:51.894815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a7a3657b-ef85-4565-b335-5818c49947f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:52.010438 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3657b-ef85-4565-b335-5818c49947f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:52.012902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3657b-ef85-4565-b335-5818c49947f6 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:52.033923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 671/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:40:56.803235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb473-c0d8-404b-9995-983159e571c0 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:56.803712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb473-c0d8-404b-9995-983159e571c0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:56.803820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb473-c0d8-404b-9995-983159e571c0 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:56.805133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 676/1347] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:40:56.813117 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171a194-7a24-4eb1-9134-e913b76732d1 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:56.816343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171a194-7a24-4eb1-9134-e913b76732d1 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:56.816343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171a194-7a24-4eb1-9134-e913b76732d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:56.830032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:56.888638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a171a194-7a24-4eb1-9134-e913b76732d1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:40:56.889823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a171a194-7a24-4eb1-9134-e913b76732d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:57.001114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 672/1348] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:40:57.286452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfb473-c0d8-404b-9995-983159e571c0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:40:57.287973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8ccfb473-c0d8-404b-9995-983159e571c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:57.346819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:57.393045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfb473-c0d8-404b-9995-983159e571c0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:40:57.466156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b5164-19c6-4349-ba37-d6cc3b04d64a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:40:57.466769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b5164-19c6-4349-ba37-d6cc3b04d64a service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:40:57.467030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b5164-19c6-4349-ba37-d6cc3b04d64a service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:40:57.467306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b5164-19c6-4349-ba37-d6cc3b04d64a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:40:57.468887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ec3b5164-19c6-4349-ba37-d6cc3b04d64a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:40:57.573260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:40:57.577486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b5164-19c6-4349-ba37-d6cc3b04d64a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:40:57.579064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b5164-19c6-4349-ba37-d6cc3b04d64a service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:40:57.597209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 677/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:02.530848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779f3c2-f7a0-4b35-89fd-c6093787738a None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:02.531723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779f3c2-f7a0-4b35-89fd-c6093787738a None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:02.533075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779f3c2-f7a0-4b35-89fd-c6093787738a None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:02.536930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 673/1350] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:41:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:41:02.542757 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bc1b0-4f6f-4dfb-afd0-d58671caea7c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:02.543211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bc1b0-4f6f-4dfb-afd0-d58671caea7c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:02.543364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bc1b0-4f6f-4dfb-afd0-d58671caea7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:02.677129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4e1bc1b0-4f6f-4dfb-afd0-d58671caea7c None None] MFA Rules not processed for user `34ec5c9c71d14762a4869b7d66e87ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:02.678527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4e1bc1b0-4f6f-4dfb-afd0-d58671caea7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:02.805047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 678/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:41:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:41:06.999657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=59 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:07.054082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e37e3c-5722-4b0f-82a1-c952721c6208 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:07.054435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e37e3c-5722-4b0f-82a1-c952721c6208 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:07.054632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e37e3c-5722-4b0f-82a1-c952721c6208 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:07.056145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 674/1352] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:07.061756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fd4f0-4801-46d9-830c-0ab8ad4ca680 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:07.062277 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fd4f0-4801-46d9-830c-0ab8ad4ca680 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:07.062503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fd4f0-4801-46d9-830c-0ab8ad4ca680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:07.118929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:07.143456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-299fd4f0-4801-46d9-830c-0ab8ad4ca680 None None] MFA Rules not processed for user `34ec5c9c71d14762a4869b7d66e87ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:07.144829 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-299fd4f0-4801-46d9-830c-0ab8ad4ca680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:07.256917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 679/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:41:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:08.555985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2f14-cdce-4471-8c08-083e3c7ce672 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:08.556536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2f14-cdce-4471-8c08-083e3c7ce672 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:08.556536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2f14-cdce-4471-8c08-083e3c7ce672 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:08.558041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 675/1354] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:08.563817 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17a9ae-cb64-4f3d-b499-12191aa3ccc6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:08.564261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17a9ae-cb64-4f3d-b499-12191aa3ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:08.564416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17a9ae-cb64-4f3d-b499-12191aa3ccc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:08.645395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-be17a9ae-cb64-4f3d-b499-12191aa3ccc6 None None] MFA Rules not processed for user `34ec5c9c71d14762a4869b7d66e87ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:08.646641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-be17a9ae-cb64-4f3d-b499-12191aa3ccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:08.689561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:08.756370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 680/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:41:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:12.425372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae46d8-7737-41c7-a599-ee75c6542376 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:12.426006 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae46d8-7737-41c7-a599-ee75c6542376 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:12.426006 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae46d8-7737-41c7-a599-ee75c6542376 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:12.427283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 676/1356] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:12.432753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c12d3-8494-41ad-ab29-fc320900a353 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:12.434243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c12d3-8494-41ad-ab29-fc320900a353 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:12.434685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c12d3-8494-41ad-ab29-fc320900a353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:12.504759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5d7c12d3-8494-41ad-ab29-fc320900a353 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:12.505905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5d7c12d3-8494-41ad-ab29-fc320900a353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:12.607965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 681/1357] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:12.920830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4fb08-cc1a-41e9-9cb9-16ca9fed0c1d None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:12.921113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4fb08-cc1a-41e9-9cb9-16ca9fed0c1d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:12.921386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4fb08-cc1a-41e9-9cb9-16ca9fed0c1d None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:12.930629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 677/1358] 10.222.0.42 () {54 vars in 965 bytes} [Sat Sep 17 03:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:12.932329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f54e13-ae85-4741-b447-ed6e556c5f8f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:12.933196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f54e13-ae85-4741-b447-ed6e556c5f8f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:12.934039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f54e13-ae85-4741-b447-ed6e556c5f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:12.959612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:13.029061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-06f54e13-ae85-4741-b447-ed6e556c5f8f None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:13.030766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-06f54e13-ae85-4741-b447-ed6e556c5f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:13.312428 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 682/1359] 10.222.0.42 () {58 vars in 1062 bytes} [Sat Sep 17 03:41:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:13.319983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4fb08-cc1a-41e9-9cb9-16ca9fed0c1d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:13.321490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ffc4fb08-cc1a-41e9-9cb9-16ca9fed0c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:13.485683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4fb08-cc1a-41e9-9cb9-16ca9fed0c1d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:13.578672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:13.623762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9f9bf-c023-40e2-8813-29cdc00de60a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:13.624370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9f9bf-c023-40e2-8813-29cdc00de60a service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:13.624647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9f9bf-c023-40e2-8813-29cdc00de60a service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:13.624911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9f9bf-c023-40e2-8813-29cdc00de60a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:13.626650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-46d9f9bf-c023-40e2-8813-29cdc00de60a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:13.723830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9f9bf-c023-40e2-8813-29cdc00de60a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:13.725410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9f9bf-c023-40e2-8813-29cdc00de60a service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:13.744961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 678/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:16.211192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1175d3a-79a1-48d9-b1c7-b9afd1808380 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:16.211752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1175d3a-79a1-48d9-b1c7-b9afd1808380 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:16.211752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1175d3a-79a1-48d9-b1c7-b9afd1808380 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:16.213360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 683/1361] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:16.218627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f89f6-e935-4156-9118-d7844d256f6d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:16.219091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f89f6-e935-4156-9118-d7844d256f6d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:16.219423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f89f6-e935-4156-9118-d7844d256f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:16.288097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-0c7f89f6-e935-4156-9118-d7844d256f6d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:16.289391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0c7f89f6-e935-4156-9118-d7844d256f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:16.329194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:16.404893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 679/1362] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:16.712000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1175d3a-79a1-48d9-b1c7-b9afd1808380 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:16.713629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d1175d3a-79a1-48d9-b1c7-b9afd1808380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:16.783837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:16.872084 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1175d3a-79a1-48d9-b1c7-b9afd1808380 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:16.991569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763527e8-e40b-4d50-96b1-5018d826749e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:16.992175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763527e8-e40b-4d50-96b1-5018d826749e service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:16.992449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763527e8-e40b-4d50-96b1-5018d826749e service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:16.992713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763527e8-e40b-4d50-96b1-5018d826749e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:16.994343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-763527e8-e40b-4d50-96b1-5018d826749e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:17.018813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:17.090329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763527e8-e40b-4d50-96b1-5018d826749e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:17.092034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763527e8-e40b-4d50-96b1-5018d826749e service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:17.112061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 684/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:19.799327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1008f-3c3c-4ad0-8930-7899c20d8568 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:19.799910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1008f-3c3c-4ad0-8930-7899c20d8568 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:19.799910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1008f-3c3c-4ad0-8930-7899c20d8568 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:19.801909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 680/1364] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:19.807824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879811-94d3-4f3b-98c1-e91bfd230c9f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:19.809063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879811-94d3-4f3b-98c1-e91bfd230c9f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:19.809063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879811-94d3-4f3b-98c1-e91bfd230c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:19.880532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3d879811-94d3-4f3b-98c1-e91bfd230c9f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:19.881775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3d879811-94d3-4f3b-98c1-e91bfd230c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:20.002398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:20.014050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 685/1365] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:20.323147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a7d37-f0b9-46d5-8127-b2b294c1115a None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:20.323421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a7d37-f0b9-46d5-8127-b2b294c1115a None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:20.324258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a7d37-f0b9-46d5-8127-b2b294c1115a None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:20.325912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 681/1366] 10.222.0.42 () {54 vars in 965 bytes} [Sat Sep 17 03:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:20.331113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07336c5-d124-4015-ace7-88e2b0322fda None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:20.331740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07336c5-d124-4015-ace7-88e2b0322fda None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:20.332687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07336c5-d124-4015-ace7-88e2b0322fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:20.414439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d07336c5-d124-4015-ace7-88e2b0322fda None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:20.415650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d07336c5-d124-4015-ace7-88e2b0322fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:20.541623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 686/1367] 10.222.0.42 () {58 vars in 1062 bytes} [Sat Sep 17 03:41:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:20.550019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a7d37-f0b9-46d5-8127-b2b294c1115a None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:20.551573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1f5a7d37-f0b9-46d5-8127-b2b294c1115a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:20.674819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddff08-843a-40eb-9b66-c3507f1a0b2e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:20.675172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddff08-843a-40eb-9b66-c3507f1a0b2e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:20.675372 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddff08-843a-40eb-9b66-c3507f1a0b2e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:20.676851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 687/1368] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:41:20.681948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895810ea-43b2-4ea7-acb6-f6ca7174ea8a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:20.682094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895810ea-43b2-4ea7-acb6-f6ca7174ea8a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:20.682325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895810ea-43b2-4ea7-acb6-f6ca7174ea8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:20.713676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a7d37-f0b9-46d5-8127-b2b294c1115a None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:20.739345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:20.749079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:20.756790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-895810ea-43b2-4ea7-acb6-f6ca7174ea8a None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:20.761994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-895810ea-43b2-4ea7-acb6-f6ca7174ea8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:20.853813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3150a-e3e8-42aa-930a-169c74b76f6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:20.854505 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3150a-e3e8-42aa-930a-169c74b76f6a service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:20.854700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3150a-e3e8-42aa-930a-169c74b76f6a service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:20.855143 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3150a-e3e8-42aa-930a-169c74b76f6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:20.857014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0be3150a-e3e8-42aa-930a-169c74b76f6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:20.880864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 688/1369] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:20.957611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3150a-e3e8-42aa-930a-169c74b76f6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:20.959507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3150a-e3e8-42aa-930a-169c74b76f6a service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:20.980775 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 682/1370] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:21.223534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8fbc-08a1-4a0c-ae8c-7a279ac8b194 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:21.225504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8fbc-08a1-4a0c-ae8c-7a279ac8b194 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:21.225504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8fbc-08a1-4a0c-ae8c-7a279ac8b194 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:21.225504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 689/1371] 10.222.0.42 () {54 vars in 964 bytes} [Sat Sep 17 03:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:21.230871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3898f6-7fae-45c1-a1a1-8f44323eea18 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:21.231346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3898f6-7fae-45c1-a1a1-8f44323eea18 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:21.231623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3898f6-7fae-45c1-a1a1-8f44323eea18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:21.268281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:21.331922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-aa3898f6-7fae-45c1-a1a1-8f44323eea18 None None] MFA Rules not processed for user `727f7207d82640a3a228dfb83814148b`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:21.333252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-aa3898f6-7fae-45c1-a1a1-8f44323eea18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:21.424533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 683/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Sat Sep 17 03:41:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:21.432361 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8fbc-08a1-4a0c-ae8c-7a279ac8b194 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:21.434444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a46a8fbc-08a1-4a0c-ae8c-7a279ac8b194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:21.539343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8fbc-08a1-4a0c-ae8c-7a279ac8b194 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:21.569140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:21.627032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed160f0-6f5b-4e2f-8bfe-2ab6afec7b3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:21.627857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed160f0-6f5b-4e2f-8bfe-2ab6afec7b3e service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:21.628304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed160f0-6f5b-4e2f-8bfe-2ab6afec7b3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:21.628737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed160f0-6f5b-4e2f-8bfe-2ab6afec7b3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:21.630547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7ed160f0-6f5b-4e2f-8bfe-2ab6afec7b3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:21.756534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed160f0-6f5b-4e2f-8bfe-2ab6afec7b3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:21.756534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed160f0-6f5b-4e2f-8bfe-2ab6afec7b3e service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:21.775487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 690/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:23.374176 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497cd50-1d3f-4df7-9acb-559ddc123dff None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:23.374716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497cd50-1d3f-4df7-9acb-559ddc123dff None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:23.374716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497cd50-1d3f-4df7-9acb-559ddc123dff None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:23.376068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 684/1374] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:23.381475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a4262-4f4a-4407-ad6d-ac7c14155fa6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:23.381930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a4262-4f4a-4407-ad6d-ac7c14155fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:23.382126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a4262-4f4a-4407-ad6d-ac7c14155fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:23.462238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-387a4262-4f4a-4407-ad6d-ac7c14155fa6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:23.463394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-387a4262-4f4a-4407-ad6d-ac7c14155fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:23.471709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:23.571313 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 691/1375] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:23.864396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497cd50-1d3f-4df7-9acb-559ddc123dff None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:23.865971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e497cd50-1d3f-4df7-9acb-559ddc123dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:23.981660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:24.020009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497cd50-1d3f-4df7-9acb-559ddc123dff None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:24.154624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd6cc6-af54-4bc2-b4d1-7a70699b9243 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:24.155232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd6cc6-af54-4bc2-b4d1-7a70699b9243 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:24.155506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd6cc6-af54-4bc2-b4d1-7a70699b9243 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:24.155777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd6cc6-af54-4bc2-b4d1-7a70699b9243 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:24.157356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-59fd6cc6-af54-4bc2-b4d1-7a70699b9243 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:24.239674 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:24.258063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd6cc6-af54-4bc2-b4d1-7a70699b9243 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:24.259336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd6cc6-af54-4bc2-b4d1-7a70699b9243 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:24.280374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 685/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:26.694253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc1160-8016-4848-a4ea-6245b4267aaf None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:26.694837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc1160-8016-4848-a4ea-6245b4267aaf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:26.694837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc1160-8016-4848-a4ea-6245b4267aaf None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:26.696305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 692/1377] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:26.702731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ed702-450b-44d5-a074-b9a0d5297c9d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:26.703134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ed702-450b-44d5-a074-b9a0d5297c9d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:26.703308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ed702-450b-44d5-a074-b9a0d5297c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:26.773884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-960ed702-450b-44d5-a074-b9a0d5297c9d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:26.775073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-960ed702-450b-44d5-a074-b9a0d5297c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:26.884759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 686/1378] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:27.177173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc1160-8016-4848-a4ea-6245b4267aaf None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:27.178923 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1efc1160-8016-4848-a4ea-6245b4267aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:27.323244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:27.335729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc1160-8016-4848-a4ea-6245b4267aaf None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:27.459658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4975234-c8b6-49f4-ad88-7c5e01f0b44d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:27.460286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4975234-c8b6-49f4-ad88-7c5e01f0b44d service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:27.460566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4975234-c8b6-49f4-ad88-7c5e01f0b44d service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:27.460836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4975234-c8b6-49f4-ad88-7c5e01f0b44d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:27.462543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c4975234-c8b6-49f4-ad88-7c5e01f0b44d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:27.569140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4975234-c8b6-49f4-ad88-7c5e01f0b44d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:27.570947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4975234-c8b6-49f4-ad88-7c5e01f0b44d service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:27.582395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:27.592771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 693/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:29.950550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7d976-3faa-4d8e-a322-39683d728d04 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:29.951376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7d976-3faa-4d8e-a322-39683d728d04 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:29.951376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7d976-3faa-4d8e-a322-39683d728d04 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:29.952451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 687/1380] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:29.958458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a537c1-0abc-47bb-9607-d85abbf70d79 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:29.959182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a537c1-0abc-47bb-9607-d85abbf70d79 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:29.959513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a537c1-0abc-47bb-9607-d85abbf70d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:30.041431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-89a537c1-0abc-47bb-9607-d85abbf70d79 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:30.043654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-89a537c1-0abc-47bb-9607-d85abbf70d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:30.166192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 694/1381] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:30.513290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7d976-3faa-4d8e-a322-39683d728d04 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:30.514888 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-11b7d976-3faa-4d8e-a322-39683d728d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:30.577831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:30.675542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7d976-3faa-4d8e-a322-39683d728d04 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:30.805556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b9a75-ff7e-427d-b5f8-f1bb8dd36211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:30.806515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b9a75-ff7e-427d-b5f8-f1bb8dd36211 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:30.806912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b9a75-ff7e-427d-b5f8-f1bb8dd36211 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:30.807301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b9a75-ff7e-427d-b5f8-f1bb8dd36211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:30.810396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1d7b9a75-ff7e-427d-b5f8-f1bb8dd36211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:30.837194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:30.928141 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b9a75-ff7e-427d-b5f8-f1bb8dd36211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:30.929886 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b9a75-ff7e-427d-b5f8-f1bb8dd36211 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:30.949554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 688/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:33.934675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a624ab-d69e-4f09-8750-b451d16a6840 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:33.935247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a624ab-d69e-4f09-8750-b451d16a6840 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:33.935413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a624ab-d69e-4f09-8750-b451d16a6840 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:33.936949 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 695/1383] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:41:33.943657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd821fe-e60a-479b-aaaf-cdf43e7647d2 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:33.944082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd821fe-e60a-479b-aaaf-cdf43e7647d2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:33.944357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd821fe-e60a-479b-aaaf-cdf43e7647d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:34.022857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ffd821fe-e60a-479b-aaaf-cdf43e7647d2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:34.024067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ffd821fe-e60a-479b-aaaf-cdf43e7647d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:34.144991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:34.154101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 689/1384] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:34.578367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a624ab-d69e-4f09-8750-b451d16a6840 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:34.579956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e9a624ab-d69e-4f09-8750-b451d16a6840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:34.662736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:34.770309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a624ab-d69e-4f09-8750-b451d16a6840 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:34.900887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131b058-ae6f-4a71-9326-3f6e9561a84d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:34.902032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131b058-ae6f-4a71-9326-3f6e9561a84d service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:34.902509 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131b058-ae6f-4a71-9326-3f6e9561a84d service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:34.902890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131b058-ae6f-4a71-9326-3f6e9561a84d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:34.904543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9131b058-ae6f-4a71-9326-3f6e9561a84d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:34.957528 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:35.024440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131b058-ae6f-4a71-9326-3f6e9561a84d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:35.026915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131b058-ae6f-4a71-9326-3f6e9561a84d service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:35.057289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 696/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:37.283363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c10804-502c-4757-8ea5-f47f8c17e5e6 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:37.283955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c10804-502c-4757-8ea5-f47f8c17e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:37.283955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c10804-502c-4757-8ea5-f47f8c17e5e6 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:37.285270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 690/1386] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:37.290916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473a118-41eb-4c61-89d6-a678723d4e85 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:37.291418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473a118-41eb-4c61-89d6-a678723d4e85 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:37.291539 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473a118-41eb-4c61-89d6-a678723d4e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:37.359916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d473a118-41eb-4c61-89d6-a678723d4e85 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:37.361125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d473a118-41eb-4c61-89d6-a678723d4e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:37.376955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c17d0-a91b-4a72-b9fe-004806e41ad0 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:37.377230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c17d0-a91b-4a72-b9fe-004806e41ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:37.377502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c17d0-a91b-4a72-b9fe-004806e41ad0 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:37.379025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 691/1387] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:41:37.383794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d641c-8662-4d9d-895b-d09363da8904 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:37.384069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d641c-8662-4d9d-895b-d09363da8904 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:37.384337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d641c-8662-4d9d-895b-d09363da8904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:37.451949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ae1d641c-8662-4d9d-895b-d09363da8904 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:37.453253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ae1d641c-8662-4d9d-895b-d09363da8904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:37.462908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 697/1388] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:37.550516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 692/1389] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:41:37.753432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473a118-41eb-4c61-89d6-a678723d4e85 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:37.754667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d473a118-41eb-4c61-89d6-a678723d4e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:37.770885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:37.863911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf44ba-bc1d-46d6-8dad-4a61a0863402 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:37.864732 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf44ba-bc1d-46d6-8dad-4a61a0863402 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:37.865017 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf44ba-bc1d-46d6-8dad-4a61a0863402 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:37.866436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 693/1390] 10.222.0.42 () {54 vars in 962 bytes} [Sat Sep 17 03:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:41:37.872340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19d66-b055-4ef5-a760-d8921c2124c0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:37.878155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19d66-b055-4ef5-a760-d8921c2124c0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:37.878155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19d66-b055-4ef5-a760-d8921c2124c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:37.878509 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473a118-41eb-4c61-89d6-a678723d4e85 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:37.954935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e1d19d66-b055-4ef5-a760-d8921c2124c0 None None] MFA Rules not processed for user `ab54ee52004746eaadb5c2970e59afc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:37.956189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e1d19d66-b055-4ef5-a760-d8921c2124c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:37.973768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:37.997525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa2cf4-0566-4489-99d6-56705e211918 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:37.998161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa2cf4-0566-4489-99d6-56705e211918 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:37.998442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa2cf4-0566-4489-99d6-56705e211918 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:37.998711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa2cf4-0566-4489-99d6-56705e211918 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:38.000285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-07aa2cf4-0566-4489-99d6-56705e211918 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:38.010074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:38.036173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 694/1391] 10.222.0.42 () {58 vars in 1059 bytes} [Sat Sep 17 03:41:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:41:38.045699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d19d66-b055-4ef5-a760-d8921c2124c0 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:38.053359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d19d66-b055-4ef5-a760-d8921c2124c0 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:38.094100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa2cf4-0566-4489-99d6-56705e211918 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:38.095712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa2cf4-0566-4489-99d6-56705e211918 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:38.114273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 698/1392] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:38.127743 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007a727-7c95-4276-9470-f08a2986f832 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab54ee52004746eaadb5c2970e59afc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:38.128348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007a727-7c95-4276-9470-f08a2986f832 service glance] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:38.128614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007a727-7c95-4276-9470-f08a2986f832 service glance] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:38.128880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007a727-7c95-4276-9470-f08a2986f832 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:38.130484 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d007a727-7c95-4276-9470-f08a2986f832 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:38.225951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:38.252753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007a727-7c95-4276-9470-f08a2986f832 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:38.254285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007a727-7c95-4276-9470-f08a2986f832 service glance] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:38.272353 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 695/1393] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:41:39.200404 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fbb9c-56d4-45c4-b595-f94808ed0256 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:39.200908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fbb9c-56d4-45c4-b595-f94808ed0256 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:39.201128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fbb9c-56d4-45c4-b595-f94808ed0256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:39.267897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7c0fbb9c-56d4-45c4-b595-f94808ed0256 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:39.269090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7c0fbb9c-56d4-45c4-b595-f94808ed0256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:39.352674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:39.356750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 699/1394] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:41:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:39.403836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b69f76-3e1d-41b9-afb0-8f3c7823de06 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:39.404323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b69f76-3e1d-41b9-afb0-8f3c7823de06 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:39.404423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b69f76-3e1d-41b9-afb0-8f3c7823de06 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:39.405763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 696/1395] 10.222.0.42 () {54 vars in 943 bytes} [Sat Sep 17 03:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:39.412326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53adf24-d7e7-45bd-9f14-393d7ba0a4f0 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:39.412613 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53adf24-d7e7-45bd-9f14-393d7ba0a4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:39.412890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53adf24-d7e7-45bd-9f14-393d7ba0a4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:39.491562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a53adf24-d7e7-45bd-9f14-393d7ba0a4f0 None None] MFA Rules not processed for user `c1136770f79a47f6a2af0b0a32259209`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:39.492771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a53adf24-d7e7-45bd-9f14-393d7ba0a4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:39.600292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 700/1396] 10.222.0.42 () {58 vars in 1040 bytes} [Sat Sep 17 03:41:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:41:39.609940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b69f76-3e1d-41b9-afb0-8f3c7823de06 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:39.611511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-19b69f76-3e1d-41b9-afb0-8f3c7823de06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:39.750108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b69f76-3e1d-41b9-afb0-8f3c7823de06 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:39.807157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:39.858093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df15746-0c8e-4634-be85-3ea26d6f4368 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:39.858606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df15746-0c8e-4634-be85-3ea26d6f4368 service swift] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:39.858882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df15746-0c8e-4634-be85-3ea26d6f4368 service swift] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:39.859152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df15746-0c8e-4634-be85-3ea26d6f4368 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:39.860751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8df15746-0c8e-4634-be85-3ea26d6f4368 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:39.955310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df15746-0c8e-4634-be85-3ea26d6f4368 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:39.956907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df15746-0c8e-4634-be85-3ea26d6f4368 service swift] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:39.965492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 697/1397] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:41:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:41:40.452596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d1b25-033c-4ff1-890e-6593e2d648ef None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:40.453099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d1b25-033c-4ff1-890e-6593e2d648ef None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:40.453099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d1b25-033c-4ff1-890e-6593e2d648ef None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:40.455532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 701/1398] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:40.460510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426c86a-c5b3-4c33-96bb-d6b10f68cf4b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:40.460928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426c86a-c5b3-4c33-96bb-d6b10f68cf4b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:40.461190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426c86a-c5b3-4c33-96bb-d6b10f68cf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:40.534309 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6426c86a-c5b3-4c33-96bb-d6b10f68cf4b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:40.535969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6426c86a-c5b3-4c33-96bb-d6b10f68cf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:40.581457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:40.650394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 698/1399] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:40.940146 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d1b25-033c-4ff1-890e-6593e2d648ef None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:40.941773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-327d1b25-033c-4ff1-890e-6593e2d648ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:41.013314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:41.112470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d1b25-033c-4ff1-890e-6593e2d648ef None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:41.239441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c54949-fdf0-460b-919a-90014ccc20ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:41.240234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c54949-fdf0-460b-919a-90014ccc20ae service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:41.240517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c54949-fdf0-460b-919a-90014ccc20ae service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:41.240793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c54949-fdf0-460b-919a-90014ccc20ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:41.242456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e7c54949-fdf0-460b-919a-90014ccc20ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:41.263439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:41.342659 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c54949-fdf0-460b-919a-90014ccc20ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:41.344291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c54949-fdf0-460b-919a-90014ccc20ae service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:41.365410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 702/1400] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:43.806462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0cdee-e22a-4a24-9b9a-fc2c611f16ba None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:43.807148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0cdee-e22a-4a24-9b9a-fc2c611f16ba None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:43.807148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0cdee-e22a-4a24-9b9a-fc2c611f16ba None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:43.809885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 699/1401] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:43.814622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171a56b-7e3f-4fd4-9674-feaca2849ef5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:43.815086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171a56b-7e3f-4fd4-9674-feaca2849ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:43.815173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171a56b-7e3f-4fd4-9674-feaca2849ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:43.892476 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e171a56b-7e3f-4fd4-9674-feaca2849ef5 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:43.894103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e171a56b-7e3f-4fd4-9674-feaca2849ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:43.999736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:44.013809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 703/1402] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:44.311241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0cdee-e22a-4a24-9b9a-fc2c611f16ba None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:44.312797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-14e0cdee-e22a-4a24-9b9a-fc2c611f16ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:44.482254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0cdee-e22a-4a24-9b9a-fc2c611f16ba None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:44.508976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:44.610145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1c964-9668-4e90-b0e1-ceeba10311dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:44.610868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1c964-9668-4e90-b0e1-ceeba10311dc service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:44.611147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1c964-9668-4e90-b0e1-ceeba10311dc service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:44.611418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1c964-9668-4e90-b0e1-ceeba10311dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:44.613029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9ae1c964-9668-4e90-b0e1-ceeba10311dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:44.723053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1c964-9668-4e90-b0e1-ceeba10311dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:44.724374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1c964-9668-4e90-b0e1-ceeba10311dc service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:44.743465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 700/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:47.163630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4deb2-4d7a-45fd-889d-f4c7175157d5 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:47.164238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4deb2-4d7a-45fd-889d-f4c7175157d5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:47.164238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4deb2-4d7a-45fd-889d-f4c7175157d5 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:47.171761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 704/1404] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:47.173527 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56045c13-014a-41a7-9201-dcf5c1a197c9 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:47.174059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56045c13-014a-41a7-9201-dcf5c1a197c9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:47.174160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56045c13-014a-41a7-9201-dcf5c1a197c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:47.233925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:47.289772 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-56045c13-014a-41a7-9201-dcf5c1a197c9 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:47.289772 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-56045c13-014a-41a7-9201-dcf5c1a197c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:47.449224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 701/1405] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:47.757072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4deb2-4d7a-45fd-889d-f4c7175157d5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:47.758622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a7d4deb2-4d7a-45fd-889d-f4c7175157d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:47.933272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4deb2-4d7a-45fd-889d-f4c7175157d5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:48.010848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:48.065362 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8648ba1-9c81-4d7d-bd34-aa0385e6d501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:48.066006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8648ba1-9c81-4d7d-bd34-aa0385e6d501 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:48.066287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8648ba1-9c81-4d7d-bd34-aa0385e6d501 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:48.066561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8648ba1-9c81-4d7d-bd34-aa0385e6d501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:48.068173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e8648ba1-9c81-4d7d-bd34-aa0385e6d501 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:48.193587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8648ba1-9c81-4d7d-bd34-aa0385e6d501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:48.195412 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8648ba1-9c81-4d7d-bd34-aa0385e6d501 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:48.219862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 705/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:41:51.350491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33a26a-cc7d-4fe3-b1f4-f9167e909409 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:51.351062 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33a26a-cc7d-4fe3-b1f4-f9167e909409 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:51.351062 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33a26a-cc7d-4fe3-b1f4-f9167e909409 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:51.352385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 702/1407] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:41:51.359153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66109e58-1fc8-4eb1-a9a9-48d96083b11f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:51.359574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66109e58-1fc8-4eb1-a9a9-48d96083b11f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:51.359753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66109e58-1fc8-4eb1-a9a9-48d96083b11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:51.437020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-66109e58-1fc8-4eb1-a9a9-48d96083b11f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:41:51.438415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-66109e58-1fc8-4eb1-a9a9-48d96083b11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:51.480083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:51.551464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 706/1408] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:41:51.853347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33a26a-cc7d-4fe3-b1f4-f9167e909409 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:41:51.854945 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-dc33a26a-cc7d-4fe3-b1f4-f9167e909409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:51.965699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:52.012085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33a26a-cc7d-4fe3-b1f4-f9167e909409 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:41:52.146995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5169f-3793-4e34-8159-2977d3a81817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:41:52.147509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5169f-3793-4e34-8159-2977d3a81817 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:41:52.147782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5169f-3793-4e34-8159-2977d3a81817 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:41:52.148051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5169f-3793-4e34-8159-2977d3a81817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:41:52.149636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b8d5169f-3793-4e34-8159-2977d3a81817 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:41:52.228856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:41:52.251631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5169f-3793-4e34-8159-2977d3a81817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:41:52.253241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5169f-3793-4e34-8159-2977d3a81817 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:41:52.273871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 703/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:01.549977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=29 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:02.270925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=14 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:04.686979 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37b21-0f59-4305-8342-520626543e86 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:04.687518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37b21-0f59-4305-8342-520626543e86 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:04.687518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37b21-0f59-4305-8342-520626543e86 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:04.688871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 707/1410] 10.222.0.42 () {54 vars in 918 bytes} [Sat Sep 17 03:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:04.694155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e0979-355a-41de-a907-512dda82b5b6 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:04.694585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e0979-355a-41de-a907-512dda82b5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:04.694818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e0979-355a-41de-a907-512dda82b5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:04.770861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-947e0979-355a-41de-a907-512dda82b5b6 None None] MFA Rules not processed for user `34ec5c9c71d14762a4869b7d66e87ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:04.772074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-947e0979-355a-41de-a907-512dda82b5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:04.878120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 704/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Sat Sep 17 03:42:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:42:04.892946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eaa9b1-26ac-4a28-813f-883a36f31392 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:04.893232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eaa9b1-26ac-4a28-813f-883a36f31392 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:04.893508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eaa9b1-26ac-4a28-813f-883a36f31392 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:04.894897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 708/1412] 10.222.0.42 () {54 vars in 918 bytes} [Sat Sep 17 03:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:04.899541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115547e-9e39-4459-a4dd-f75065f22bf4 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:04.899823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115547e-9e39-4459-a4dd-f75065f22bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:04.900095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115547e-9e39-4459-a4dd-f75065f22bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:04.973812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:04.976707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e115547e-9e39-4459-a4dd-f75065f22bf4 None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:04.977934 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e115547e-9e39-4459-a4dd-f75065f22bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:05.100880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 705/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Sat Sep 17 03:42:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:07.726042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea869-27f5-4acc-99d8-09f52fa80965 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:07.726654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea869-27f5-4acc-99d8-09f52fa80965 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:07.726768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea869-27f5-4acc-99d8-09f52fa80965 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:07.728045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:07.734244 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d7db-995f-47ea-9570-1e36cf17f761 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:07.734701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d7db-995f-47ea-9570-1e36cf17f761 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:07.734824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d7db-995f-47ea-9570-1e36cf17f761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:07.805044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-c538d7db-995f-47ea-9570-1e36cf17f761 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:07.806287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c538d7db-995f-47ea-9570-1e36cf17f761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:07.863769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:07.907443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:08.200282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ea869-27f5-4acc-99d8-09f52fa80965 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:08.201829 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5d1ea869-27f5-4acc-99d8-09f52fa80965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:08.353560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ea869-27f5-4acc-99d8-09f52fa80965 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:08.431300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:08.469237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a3ce0-b716-4e32-956d-2a485129f8d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:08.469882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a3ce0-b716-4e32-956d-2a485129f8d5 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:08.470163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a3ce0-b716-4e32-956d-2a485129f8d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:08.470433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a3ce0-b716-4e32-956d-2a485129f8d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:08.472008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-539a3ce0-b716-4e32-956d-2a485129f8d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:08.564672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a3ce0-b716-4e32-956d-2a485129f8d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:08.566400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a3ce0-b716-4e32-956d-2a485129f8d5 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:08.584916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 710/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:09.755724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb96bde-0664-4cf3-982c-81840e78939d None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:09.756292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb96bde-0664-4cf3-982c-81840e78939d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:09.756292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb96bde-0664-4cf3-982c-81840e78939d None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:09.757607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:09.764616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6a1c7-be34-4c6f-acd0-08ba53d1ae18 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:09.765055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6a1c7-be34-4c6f-acd0-08ba53d1ae18 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:09.765152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6a1c7-be34-4c6f-acd0-08ba53d1ae18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:09.841397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6ef6a1c7-be34-4c6f-acd0-08ba53d1ae18 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:09.842560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6ef6a1c7-be34-4c6f-acd0-08ba53d1ae18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:09.869296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:09.938773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:10.090510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb96bde-0664-4cf3-982c-81840e78939d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:10.092100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-2bb96bde-0664-4cf3-982c-81840e78939d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:10.240627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb96bde-0664-4cf3-982c-81840e78939d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:10.280389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:10.351878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb1c9f-5097-48eb-bac1-3129f75a28ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:10.352397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb1c9f-5097-48eb-bac1-3129f75a28ee service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:10.352672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb1c9f-5097-48eb-bac1-3129f75a28ee service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:10.352940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb1c9f-5097-48eb-bac1-3129f75a28ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:10.354709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9beb1c9f-5097-48eb-bac1-3129f75a28ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:10.445021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb1c9f-5097-48eb-bac1-3129f75a28ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:10.446640 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb1c9f-5097-48eb-bac1-3129f75a28ee service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:10.464815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 708/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:11.606615 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f5f42-c366-43ee-8c29-d0555432445b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:11.607190 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f5f42-c366-43ee-8c29-d0555432445b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:11.607190 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f5f42-c366-43ee-8c29-d0555432445b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:11.608542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:42:11.614971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d1d89-9a98-400f-83c4-6d82b3d15aa1 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:11.615445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d1d89-9a98-400f-83c4-6d82b3d15aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:11.615598 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d1d89-9a98-400f-83c4-6d82b3d15aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:11.670839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:11.692270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-8b2d1d89-9a98-400f-83c4-6d82b3d15aa1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:11.693465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8b2d1d89-9a98-400f-83c4-6d82b3d15aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:11.791044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:11.946863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f5f42-c366-43ee-8c29-d0555432445b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:11.949001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ff0f5f42-c366-43ee-8c29-d0555432445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:12.096634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f5f42-c366-43ee-8c29-d0555432445b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:12.109489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:12.211417 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de3a49-ce2c-496a-8786-ef236afe102a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:12.211933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de3a49-ce2c-496a-8786-ef236afe102a service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:12.212208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de3a49-ce2c-496a-8786-ef236afe102a service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:12.212477 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de3a49-ce2c-496a-8786-ef236afe102a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:12.214280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-85de3a49-ce2c-496a-8786-ef236afe102a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:12.301469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de3a49-ce2c-496a-8786-ef236afe102a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:12.303083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de3a49-ce2c-496a-8786-ef236afe102a service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:12.321629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 713/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:42:13.473069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cffa44-b8de-45f4-b3e8-8bcaf6a4fb10 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:13.473607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cffa44-b8de-45f4-b3e8-8bcaf6a4fb10 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:13.473607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cffa44-b8de-45f4-b3e8-8bcaf6a4fb10 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:13.474991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:42:13.481191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef249d2-b340-4584-9939-0e633e9218f3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:13.481678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef249d2-b340-4584-9939-0e633e9218f3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:13.481800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef249d2-b340-4584-9939-0e633e9218f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:13.502144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:13.560306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-aef249d2-b340-4584-9939-0e633e9218f3 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:13.561519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-aef249d2-b340-4584-9939-0e633e9218f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:13.656864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:13.810638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cffa44-b8de-45f4-b3e8-8bcaf6a4fb10 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:13.812112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f5cffa44-b8de-45f4-b3e8-8bcaf6a4fb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:13.920584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:13.956133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cffa44-b8de-45f4-b3e8-8bcaf6a4fb10 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:14.068847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2d5a0-c806-4d60-9ca3-0c6704ee8bd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:14.069376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2d5a0-c806-4d60-9ca3-0c6704ee8bd8 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:14.069654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2d5a0-c806-4d60-9ca3-0c6704ee8bd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:14.069955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2d5a0-c806-4d60-9ca3-0c6704ee8bd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:14.071525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a1e2d5a0-c806-4d60-9ca3-0c6704ee8bd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:14.149454 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:14.163422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2d5a0-c806-4d60-9ca3-0c6704ee8bd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:14.165031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2d5a0-c806-4d60-9ca3-0c6704ee8bd8 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:14.184386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 711/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:15.349088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff810b7-f218-47d3-93f5-fcc55d00a158 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:15.349680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff810b7-f218-47d3-93f5-fcc55d00a158 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:15.349680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff810b7-f218-47d3-93f5-fcc55d00a158 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:15.351105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:15.358059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b633a3-aa3e-4bc2-8248-57a1ccd512bf None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:15.358531 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b633a3-aa3e-4bc2-8248-57a1ccd512bf None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:15.358642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b633a3-aa3e-4bc2-8248-57a1ccd512bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:15.433884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e8b633a3-aa3e-4bc2-8248-57a1ccd512bf None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:15.435094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e8b633a3-aa3e-4bc2-8248-57a1ccd512bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:15.532020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:15.682173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff810b7-f218-47d3-93f5-fcc55d00a158 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:15.684393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-aff810b7-f218-47d3-93f5-fcc55d00a158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:15.766655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:15.831180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff810b7-f218-47d3-93f5-fcc55d00a158 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:15.945179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72264f71-418c-4c87-ad7a-207705b97542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:15.945833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72264f71-418c-4c87-ad7a-207705b97542 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:15.946101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72264f71-418c-4c87-ad7a-207705b97542 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:15.946374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72264f71-418c-4c87-ad7a-207705b97542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:15.948477 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-72264f71-418c-4c87-ad7a-207705b97542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:15.995926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:16.036955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72264f71-418c-4c87-ad7a-207705b97542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:16.038618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72264f71-418c-4c87-ad7a-207705b97542 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:16.056738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 716/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:17.169358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc35a2-dfaa-42dd-ab9a-d7f1cdcf732d None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:17.170011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc35a2-dfaa-42dd-ab9a-d7f1cdcf732d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:17.170011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc35a2-dfaa-42dd-ab9a-d7f1cdcf732d None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:17.171261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:17.177439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee505c-4abf-41fb-8781-e68e507eb4a0 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:17.177902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee505c-4abf-41fb-8781-e68e507eb4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:17.178014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee505c-4abf-41fb-8781-e68e507eb4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:17.254628 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-81ee505c-4abf-41fb-8781-e68e507eb4a0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:17.255852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-81ee505c-4abf-41fb-8781-e68e507eb4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:17.350032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:17.501858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc35a2-dfaa-42dd-ab9a-d7f1cdcf732d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:17.503393 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-fafc35a2-dfaa-42dd-ab9a-d7f1cdcf732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:17.552766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:17.650171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc35a2-dfaa-42dd-ab9a-d7f1cdcf732d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:17.763411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44176bb-9b91-40da-981e-230b36209938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:17.763939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44176bb-9b91-40da-981e-230b36209938 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:17.764217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44176bb-9b91-40da-981e-230b36209938 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:17.764488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44176bb-9b91-40da-981e-230b36209938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:17.766108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c44176bb-9b91-40da-981e-230b36209938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:17.772908 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:17.866371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44176bb-9b91-40da-981e-230b36209938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:17.867968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44176bb-9b91-40da-981e-230b36209938 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:17.887025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 714/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:19.016578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4066d-03ea-4b1f-8f24-a4a86180065b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:19.017164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4066d-03ea-4b1f-8f24-a4a86180065b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:19.017164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4066d-03ea-4b1f-8f24-a4a86180065b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:19.018523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:19.024667 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf6fb-8f2a-4d50-b4e4-f45fc6e4be60 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:19.025133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf6fb-8f2a-4d50-b4e4-f45fc6e4be60 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:19.025239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf6fb-8f2a-4d50-b4e4-f45fc6e4be60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:19.101122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-843bf6fb-8f2a-4d50-b4e4-f45fc6e4be60 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:19.102300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-843bf6fb-8f2a-4d50-b4e4-f45fc6e4be60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:19.169662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:19.204042 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:19.356438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4066d-03ea-4b1f-8f24-a4a86180065b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:19.357911 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-70a4066d-03ea-4b1f-8f24-a4a86180065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:19.375274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:19.502566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4066d-03ea-4b1f-8f24-a4a86180065b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:19.593188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:19.619942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38871915-930f-49ed-83a8-1db4d6643826 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:19.620470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38871915-930f-49ed-83a8-1db4d6643826 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:19.620745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38871915-930f-49ed-83a8-1db4d6643826 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:19.621014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38871915-930f-49ed-83a8-1db4d6643826 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:19.622673 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-38871915-930f-49ed-83a8-1db4d6643826 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:19.712393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38871915-930f-49ed-83a8-1db4d6643826 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:19.714598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38871915-930f-49ed-83a8-1db4d6643826 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:19.732515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 719/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:20.842969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa49f7-0426-4082-b770-c5351b8fc675 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:20.843468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa49f7-0426-4082-b770-c5351b8fc675 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:20.843586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa49f7-0426-4082-b770-c5351b8fc675 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:20.844847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:20.851123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574ef12-eb16-4ecc-b08f-0cc5b4aa621e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:20.851543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574ef12-eb16-4ecc-b08f-0cc5b4aa621e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:20.851683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574ef12-eb16-4ecc-b08f-0cc5b4aa621e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:20.933398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2574ef12-eb16-4ecc-b08f-0cc5b4aa621e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:20.934569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2574ef12-eb16-4ecc-b08f-0cc5b4aa621e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:20.982369 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:21.036828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:21.189851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa49f7-0426-4082-b770-c5351b8fc675 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:21.191306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b2fa49f7-0426-4082-b770-c5351b8fc675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:21.337698 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa49f7-0426-4082-b770-c5351b8fc675 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:21.391492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:21.448589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c367d-8aaf-4996-b2d5-415d48189f42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:21.449132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c367d-8aaf-4996-b2d5-415d48189f42 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:21.449405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c367d-8aaf-4996-b2d5-415d48189f42 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:21.449672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c367d-8aaf-4996-b2d5-415d48189f42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:21.451266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-3f7c367d-8aaf-4996-b2d5-415d48189f42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:21.542394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c367d-8aaf-4996-b2d5-415d48189f42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:21.544017 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c367d-8aaf-4996-b2d5-415d48189f42 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:21.562252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 717/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:22.699775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63fddc-be83-4308-9787-0bcc324e7caf None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:22.700256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63fddc-be83-4308-9787-0bcc324e7caf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:22.700372 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63fddc-be83-4308-9787-0bcc324e7caf None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:22.701686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:22.708209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13074-2b6c-4917-bd7f-9b7e4b47f9d2 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:22.708634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13074-2b6c-4917-bd7f-9b7e4b47f9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:22.708749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13074-2b6c-4917-bd7f-9b7e4b47f9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:22.785841 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-51b13074-2b6c-4917-bd7f-9b7e4b47f9d2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:22.787030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-51b13074-2b6c-4917-bd7f-9b7e4b47f9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:22.788998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:22.886390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:42:23.036419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63fddc-be83-4308-9787-0bcc324e7caf None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:23.037890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8d63fddc-be83-4308-9787-0bcc324e7caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:23.185517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63fddc-be83-4308-9787-0bcc324e7caf None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:23.212209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:23.298748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b359735-d9d3-4f2c-a869-dca0930c5b9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:23.299261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b359735-d9d3-4f2c-a869-dca0930c5b9b service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:23.299536 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b359735-d9d3-4f2c-a869-dca0930c5b9b service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:23.299806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b359735-d9d3-4f2c-a869-dca0930c5b9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:23.301368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3b359735-d9d3-4f2c-a869-dca0930c5b9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:23.388759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b359735-d9d3-4f2c-a869-dca0930c5b9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:23.390360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b359735-d9d3-4f2c-a869-dca0930c5b9b service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:23.408549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 722/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:25.507383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b122c97-a35b-4ee7-9401-5153a087a326 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:25.507954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b122c97-a35b-4ee7-9401-5153a087a326 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:25.507954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b122c97-a35b-4ee7-9401-5153a087a326 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:25.509292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:42:25.515377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730d9e2-a9f7-4019-bbf4-fb97ccb095ec None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:25.515784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730d9e2-a9f7-4019-bbf4-fb97ccb095ec None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:25.516006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730d9e2-a9f7-4019-bbf4-fb97ccb095ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:25.556405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:25.581156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b730d9e2-a9f7-4019-bbf4-fb97ccb095ec None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:25.582370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b730d9e2-a9f7-4019-bbf4-fb97ccb095ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:25.679679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:25.975973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b122c97-a35b-4ee7-9401-5153a087a326 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:25.977491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4b122c97-a35b-4ee7-9401-5153a087a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:26.080780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b122c97-a35b-4ee7-9401-5153a087a326 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:26.115188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:26.157428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcf577-be3d-4dde-b4a4-e305f60c04d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:26.158061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcf577-be3d-4dde-b4a4-e305f60c04d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:26.158336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcf577-be3d-4dde-b4a4-e305f60c04d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:26.158609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcf577-be3d-4dde-b4a4-e305f60c04d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:26.160171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9bdcf577-be3d-4dde-b4a4-e305f60c04d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:26.268219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcf577-be3d-4dde-b4a4-e305f60c04d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:26.269637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcf577-be3d-4dde-b4a4-e305f60c04d4 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:26.288250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 720/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:28.596682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42183b58-6e7a-4b45-8cfe-fe52f832a870 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:28.597259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42183b58-6e7a-4b45-8cfe-fe52f832a870 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:28.597259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42183b58-6e7a-4b45-8cfe-fe52f832a870 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:28.598640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:28.604558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaca84-76b3-4607-8b35-3dd18790b276 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:28.605258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaca84-76b3-4607-8b35-3dd18790b276 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:28.605258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaca84-76b3-4607-8b35-3dd18790b276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:28.671622 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-abfaca84-76b3-4607-8b35-3dd18790b276 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:28.672849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-abfaca84-76b3-4607-8b35-3dd18790b276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:28.686616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:28.769953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:29.058071 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42183b58-6e7a-4b45-8cfe-fe52f832a870 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:29.059604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-42183b58-6e7a-4b45-8cfe-fe52f832a870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:29.165015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42183b58-6e7a-4b45-8cfe-fe52f832a870 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:29.178315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:29.241813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb492a-306a-4573-b396-8d17dbc855e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:29.242420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb492a-306a-4573-b396-8d17dbc855e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:29.242700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb492a-306a-4573-b396-8d17dbc855e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:29.242971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb492a-306a-4573-b396-8d17dbc855e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:29.244889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e1fb492a-306a-4573-b396-8d17dbc855e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:29.356546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb492a-306a-4573-b396-8d17dbc855e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:29.357928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb492a-306a-4573-b396-8d17dbc855e2 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:29.375584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 725/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:31.858246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6366f8-786d-4d3e-a541-5955ebfbe6be None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:31.858807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6366f8-786d-4d3e-a541-5955ebfbe6be None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:31.858807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6366f8-786d-4d3e-a541-5955ebfbe6be None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:31.860138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:31.865369 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e404d0-b949-4bcc-a7c6-f04dd2adab77 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:31.865781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e404d0-b949-4bcc-a7c6-f04dd2adab77 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:31.865959 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e404d0-b949-4bcc-a7c6-f04dd2adab77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:31.913905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:31.931996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e8e404d0-b949-4bcc-a7c6-f04dd2adab77 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:31.933278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e8e404d0-b949-4bcc-a7c6-f04dd2adab77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:32.029527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:32.316530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6366f8-786d-4d3e-a541-5955ebfbe6be None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:32.318117 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ee6366f8-786d-4d3e-a541-5955ebfbe6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:32.421908 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6366f8-786d-4d3e-a541-5955ebfbe6be None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:32.462050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:32.502314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848eef1b-5658-4e25-a542-26dc22629961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:32.502909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eef1b-5658-4e25-a542-26dc22629961 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:32.503181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eef1b-5658-4e25-a542-26dc22629961 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:32.503446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eef1b-5658-4e25-a542-26dc22629961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:32.505021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-848eef1b-5658-4e25-a542-26dc22629961 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:32.612340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848eef1b-5658-4e25-a542-26dc22629961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:32.613830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848eef1b-5658-4e25-a542-26dc22629961 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:32.632758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 723/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:34.980950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497c418-ab6b-41b2-bca0-e9942eca3fe2 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:34.981462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497c418-ab6b-41b2-bca0-e9942eca3fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:34.981462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497c418-ab6b-41b2-bca0-e9942eca3fe2 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:34.982782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:34.988023 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d80c1-042b-49b0-88d5-bb8062b67f5c None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:34.988468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d80c1-042b-49b0-88d5-bb8062b67f5c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:34.988565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d80c1-042b-49b0-88d5-bb8062b67f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:35.054250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-7b6d80c1-042b-49b0-88d5-bb8062b67f5c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:35.055445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7b6d80c1-042b-49b0-88d5-bb8062b67f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:35.075550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:35.149897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:35.435902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497c418-ab6b-41b2-bca0-e9942eca3fe2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:35.437418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1497c418-ab6b-41b2-bca0-e9942eca3fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:35.538788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497c418-ab6b-41b2-bca0-e9942eca3fe2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:35.558606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:35.613129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c778e42-cc8a-4dcf-b4fc-eb216b9ae751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:35.613779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c778e42-cc8a-4dcf-b4fc-eb216b9ae751 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:35.614061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c778e42-cc8a-4dcf-b4fc-eb216b9ae751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:35.614334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c778e42-cc8a-4dcf-b4fc-eb216b9ae751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:35.615917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1c778e42-cc8a-4dcf-b4fc-eb216b9ae751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:35.723286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c778e42-cc8a-4dcf-b4fc-eb216b9ae751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:35.724629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c778e42-cc8a-4dcf-b4fc-eb216b9ae751 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:35.742196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 728/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:38.001388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a4375-ae47-411b-b813-88f2a0cc2064 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:38.001951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a4375-ae47-411b-b813-88f2a0cc2064 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:38.001951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a4375-ae47-411b-b813-88f2a0cc2064 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:38.003305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:38.008623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72374bad-7699-4ab9-bcba-2c8db85bc609 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:38.009037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72374bad-7699-4ab9-bcba-2c8db85bc609 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:38.009202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72374bad-7699-4ab9-bcba-2c8db85bc609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:38.072962 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:38.075877 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-72374bad-7699-4ab9-bcba-2c8db85bc609 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:38.077081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-72374bad-7699-4ab9-bcba-2c8db85bc609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:38.173878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:38.466020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a4375-ae47-411b-b813-88f2a0cc2064 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:38.477343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e61a4375-ae47-411b-b813-88f2a0cc2064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:38.576359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a4375-ae47-411b-b813-88f2a0cc2064 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:38.625800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:38.650115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97731b-e0e3-45d6-b221-1553b60d9f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:38.650854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97731b-e0e3-45d6-b221-1553b60d9f9d service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:38.651134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97731b-e0e3-45d6-b221-1553b60d9f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:38.651423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97731b-e0e3-45d6-b221-1553b60d9f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:38.653066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5d97731b-e0e3-45d6-b221-1553b60d9f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:38.764267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97731b-e0e3-45d6-b221-1553b60d9f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:38.765620 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97731b-e0e3-45d6-b221-1553b60d9f9d service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:38.783689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 726/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:40.988508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1de940-42c9-46e7-8079-f7512217e81e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:40.989037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1de940-42c9-46e7-8079-f7512217e81e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:40.989037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1de940-42c9-46e7-8079-f7512217e81e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:40.990345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:40.996501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04563127-33e2-4696-b1f4-e11a45438558 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:40.996933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04563127-33e2-4696-b1f4-e11a45438558 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:40.997048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04563127-33e2-4696-b1f4-e11a45438558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:41.062691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-04563127-33e2-4696-b1f4-e11a45438558 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:41.063905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-04563127-33e2-4696-b1f4-e11a45438558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:41.094107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:41.161085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:42:41.450399 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1de940-42c9-46e7-8079-f7512217e81e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:41.452754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3c1de940-42c9-46e7-8079-f7512217e81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:41.560113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1de940-42c9-46e7-8079-f7512217e81e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:41.589098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:41.634586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c3841-301d-46a6-afa0-1a25c492cde5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:41.635744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c3841-301d-46a6-afa0-1a25c492cde5 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:41.636024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c3841-301d-46a6-afa0-1a25c492cde5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:41.636291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c3841-301d-46a6-afa0-1a25c492cde5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:41.637904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-af1c3841-301d-46a6-afa0-1a25c492cde5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:41.745943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c3841-301d-46a6-afa0-1a25c492cde5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:41.747363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c3841-301d-46a6-afa0-1a25c492cde5 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:41.765453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 731/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:43.993878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df92c8-9d14-4f94-8dde-7e0c9d00789a None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:43.995074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df92c8-9d14-4f94-8dde-7e0c9d00789a None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:43.995188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df92c8-9d14-4f94-8dde-7e0c9d00789a None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:43.997275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:44.002744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4949d5-a829-42a7-8f8b-fdb655d8cebf None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:44.003183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4949d5-a829-42a7-8f8b-fdb655d8cebf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:44.003296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4949d5-a829-42a7-8f8b-fdb655d8cebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:44.068081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3c4949d5-a829-42a7-8f8b-fdb655d8cebf None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:44.069204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3c4949d5-a829-42a7-8f8b-fdb655d8cebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:44.078081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:44.168103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:44.455299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df92c8-9d14-4f94-8dde-7e0c9d00789a None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:44.456852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-40df92c8-9d14-4f94-8dde-7e0c9d00789a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:44.568957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df92c8-9d14-4f94-8dde-7e0c9d00789a None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:44.626259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:44.644505 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ca037-6f69-4bec-af77-5ede21bdc1d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:44.645106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca037-6f69-4bec-af77-5ede21bdc1d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:44.645379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca037-6f69-4bec-af77-5ede21bdc1d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:44.645699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca037-6f69-4bec-af77-5ede21bdc1d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:44.647311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bd6ca037-6f69-4bec-af77-5ede21bdc1d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:44.757240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ca037-6f69-4bec-af77-5ede21bdc1d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:44.758618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ca037-6f69-4bec-af77-5ede21bdc1d2 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:44.777150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 729/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:47.006969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd20bfe-683f-4b3d-8cda-e41f3c772c3d None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:47.007534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd20bfe-683f-4b3d-8cda-e41f3c772c3d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:47.007534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd20bfe-683f-4b3d-8cda-e41f3c772c3d None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:47.008781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:47.014266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86505d84-c37b-404f-b327-a55e4d4f8bec None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:47.014719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86505d84-c37b-404f-b327-a55e4d4f8bec None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:47.014826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86505d84-c37b-404f-b327-a55e4d4f8bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:47.080605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-86505d84-c37b-404f-b327-a55e4d4f8bec None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:47.081852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-86505d84-c37b-404f-b327-a55e4d4f8bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:47.121163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:47.178942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:42:47.461287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd20bfe-683f-4b3d-8cda-e41f3c772c3d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:47.462922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8bd20bfe-683f-4b3d-8cda-e41f3c772c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:47.566732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd20bfe-683f-4b3d-8cda-e41f3c772c3d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:47.604007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:47.640364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe9755-8ea0-454f-8ee8-3f619d63dee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:47.640965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe9755-8ea0-454f-8ee8-3f619d63dee8 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:47.641248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe9755-8ea0-454f-8ee8-3f619d63dee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:47.641525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe9755-8ea0-454f-8ee8-3f619d63dee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:47.643148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a4fe9755-8ea0-454f-8ee8-3f619d63dee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:47.756480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe9755-8ea0-454f-8ee8-3f619d63dee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:47.757941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe9755-8ea0-454f-8ee8-3f619d63dee8 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:47.777769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 734/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:50.087130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15549ffd-8c7f-4b65-be77-2422ac3d3bc9 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:50.087647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15549ffd-8c7f-4b65-be77-2422ac3d3bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:50.087647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15549ffd-8c7f-4b65-be77-2422ac3d3bc9 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:50.090336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:50.095881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879636a-a735-4d79-8a22-0da075e2c98e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:50.096333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879636a-a735-4d79-8a22-0da075e2c98e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:50.096454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879636a-a735-4d79-8a22-0da075e2c98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:50.161004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b879636a-a735-4d79-8a22-0da075e2c98e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:50.162236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b879636a-a735-4d79-8a22-0da075e2c98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:50.180500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:50.261862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:50.545037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15549ffd-8c7f-4b65-be77-2422ac3d3bc9 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:50.546640 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-15549ffd-8c7f-4b65-be77-2422ac3d3bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:50.663890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15549ffd-8c7f-4b65-be77-2422ac3d3bc9 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:50.731116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:50.737673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831e14d-b33d-4574-8022-c52c840edd22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:50.738307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831e14d-b33d-4574-8022-c52c840edd22 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:50.738614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831e14d-b33d-4574-8022-c52c840edd22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:50.738884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831e14d-b33d-4574-8022-c52c840edd22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:50.740528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b831e14d-b33d-4574-8022-c52c840edd22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:50.849161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831e14d-b33d-4574-8022-c52c840edd22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:50.850627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831e14d-b33d-4574-8022-c52c840edd22 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:50.868713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 732/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:42:53.049025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fedf6-24dc-4850-b9f7-f34d5589deea None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:53.049514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fedf6-24dc-4850-b9f7-f34d5589deea None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:53.049617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fedf6-24dc-4850-b9f7-f34d5589deea None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:53.050946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:42:53.056279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2a6df-b77b-45d0-b7c7-00c4536c5ffa None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:53.057230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2a6df-b77b-45d0-b7c7-00c4536c5ffa None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:53.057336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2a6df-b77b-45d0-b7c7-00c4536c5ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:53.123701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-12f2a6df-b77b-45d0-b7c7-00c4536c5ffa None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:42:53.128568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-12f2a6df-b77b-45d0-b7c7-00c4536c5ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:53.178691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:53.227086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:42:53.511295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fedf6-24dc-4850-b9f7-f34d5589deea None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:42:53.512860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fb3fedf6-24dc-4850-b9f7-f34d5589deea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:53.616100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fedf6-24dc-4850-b9f7-f34d5589deea None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:42:53.663754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:42:53.688782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d862e-cefe-47ac-acc3-76b5399dbd47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:42:53.689381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d862e-cefe-47ac-acc3-76b5399dbd47 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:42:53.689669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d862e-cefe-47ac-acc3-76b5399dbd47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:42:53.689966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d862e-cefe-47ac-acc3-76b5399dbd47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:42:53.691555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d85d862e-cefe-47ac-acc3-76b5399dbd47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:42:53.802376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d862e-cefe-47ac-acc3-76b5399dbd47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:42:53.803810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d862e-cefe-47ac-acc3-76b5399dbd47 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:42:53.823994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 737/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:43:03.261808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=22 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:03.820834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=66 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:14.056533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837e1b5-0272-4b14-a2fc-ee7889c3e19f None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:14.057052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837e1b5-0272-4b14-a2fc-ee7889c3e19f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:14.057052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837e1b5-0272-4b14-a2fc-ee7889c3e19f None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:14.058385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 734/1471] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:43:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:43:14.063586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caadfdd-97a3-4687-91db-3194c055aa2a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:14.063974 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caadfdd-97a3-4687-91db-3194c055aa2a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:14.064132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caadfdd-97a3-4687-91db-3194c055aa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:14.128982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5caadfdd-97a3-4687-91db-3194c055aa2a None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:43:14.130202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5caadfdd-97a3-4687-91db-3194c055aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:14.227336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 738/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:43:14.528102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4ca49-0934-44b7-b5a8-599510e19e4c None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:14.528392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4ca49-0934-44b7-b5a8-599510e19e4c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:14.528672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4ca49-0934-44b7-b5a8-599510e19e4c None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:14.530054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 735/1473] 10.222.0.42 () {54 vars in 962 bytes} [Sat Sep 17 03:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:43:14.534952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833c3ee-d319-4406-9954-e75fdce2f507 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:14.535239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833c3ee-d319-4406-9954-e75fdce2f507 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:14.535535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833c3ee-d319-4406-9954-e75fdce2f507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:14.614281 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8833c3ee-d319-4406-9954-e75fdce2f507 None None] MFA Rules not processed for user `ab54ee52004746eaadb5c2970e59afc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:43:14.615471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8833c3ee-d319-4406-9954-e75fdce2f507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:14.637317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:14.698192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 739/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Sat Sep 17 03:43:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:43:14.708516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4ca49-0934-44b7-b5a8-599510e19e4c None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:43:14.710346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bee4ca49-0934-44b7-b5a8-599510e19e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:14.813984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4ca49-0934-44b7-b5a8-599510e19e4c None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:43:14.888152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e42ba-89e8-464b-b819-29d9a1150be5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab54ee52004746eaadb5c2970e59afc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:43:14.888742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e42ba-89e8-464b-b819-29d9a1150be5 service glance] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:14.889016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e42ba-89e8-464b-b819-29d9a1150be5 service glance] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:14.889283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e42ba-89e8-464b-b819-29d9a1150be5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:14.890909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-658e42ba-89e8-464b-b819-29d9a1150be5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:14.944327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:14.999468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e42ba-89e8-464b-b819-29d9a1150be5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:43:15.000929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e42ba-89e8-464b-b819-29d9a1150be5 service glance] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:43:15.019204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 736/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:43:17.291034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0bd0-c57b-4036-b77d-73e268bf4dd2 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:17.291881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0bd0-c57b-4036-b77d-73e268bf4dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:17.291881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0bd0-c57b-4036-b77d-73e268bf4dd2 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:17.292910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 740/1476] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:43:17.298145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4041533-08f6-43d9-a29e-2d4b597600b0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:17.298605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4041533-08f6-43d9-a29e-2d4b597600b0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:17.298718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4041533-08f6-43d9-a29e-2d4b597600b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:17.364371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d4041533-08f6-43d9-a29e-2d4b597600b0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:43:17.365584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d4041533-08f6-43d9-a29e-2d4b597600b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:17.461064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 737/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:43:17.734929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0bd0-c57b-4036-b77d-73e268bf4dd2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:43:17.736467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b0fe0bd0-c57b-4036-b77d-73e268bf4dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:17.864375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0bd0-c57b-4036-b77d-73e268bf4dd2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:43:17.916398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:17.963074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa0a36-2693-4529-9461-cc95c81bf883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:43:17.963708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa0a36-2693-4529-9461-cc95c81bf883 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:17.963991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa0a36-2693-4529-9461-cc95c81bf883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:17.964294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa0a36-2693-4529-9461-cc95c81bf883 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:17.964934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa0a36-2693-4529-9461-cc95c81bf883 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:43:17.966745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-49aa0a36-2693-4529-9461-cc95c81bf883 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:43:17.966745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Sep 17 03:43:17.966745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: Sep 17 03:43:17.967903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa0a36-2693-4529-9461-cc95c81bf883 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:43:17.987479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 741/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Sep 17 03:43:17 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:43:20.173784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e5ac7-8e58-4b4e-8a25-15d7a53abf5b None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:20.174391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e5ac7-8e58-4b4e-8a25-15d7a53abf5b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:20.174391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e5ac7-8e58-4b4e-8a25-15d7a53abf5b None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:20.175674 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 738/1479] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:43:20.181825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c509ff1-5c7f-435e-9eee-a4a4d607b06b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:20.182259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c509ff1-5c7f-435e-9eee-a4a4d607b06b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:20.182368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c509ff1-5c7f-435e-9eee-a4a4d607b06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:20.248123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4c509ff1-5c7f-435e-9eee-a4a4d607b06b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:43:20.249329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4c509ff1-5c7f-435e-9eee-a4a4d607b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:20.344938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 742/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:43:20.641708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e5ac7-8e58-4b4e-8a25-15d7a53abf5b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:43:20.643271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a38e5ac7-8e58-4b4e-8a25-15d7a53abf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:20.653634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:20.793423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e5ac7-8e58-4b4e-8a25-15d7a53abf5b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:43:20.874180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:20.907327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684d347-6b5d-4e70-bc4d-5cd14c62e867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:43:20.907927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d347-6b5d-4e70-bc4d-5cd14c62e867 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:20.908202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d347-6b5d-4e70-bc4d-5cd14c62e867 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:20.908472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d347-6b5d-4e70-bc4d-5cd14c62e867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:20.910159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-2684d347-6b5d-4e70-bc4d-5cd14c62e867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:21.000026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684d347-6b5d-4e70-bc4d-5cd14c62e867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:43:21.001627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684d347-6b5d-4e70-bc4d-5cd14c62e867 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:43:21.019677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 739/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:43:23.454942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10deafa-a436-41ec-8a33-558322a4192f None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:23.455482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10deafa-a436-41ec-8a33-558322a4192f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:23.455482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10deafa-a436-41ec-8a33-558322a4192f None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:23.456904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 743/1482] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:43:23.462746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3efb9-e574-43ad-ac8d-297e3ff1650b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:23.463216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3efb9-e574-43ad-ac8d-297e3ff1650b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:23.463322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3efb9-e574-43ad-ac8d-297e3ff1650b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:23.528118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-43d3efb9-e574-43ad-ac8d-297e3ff1650b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:43:23.529324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-43d3efb9-e574-43ad-ac8d-297e3ff1650b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:23.574823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:23.626074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 740/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:43:23.932726 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10deafa-a436-41ec-8a33-558322a4192f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:43:23.934298 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f10deafa-a436-41ec-8a33-558322a4192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:23.961798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:24.044572 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10deafa-a436-41ec-8a33-558322a4192f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:43:24.119929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80342798-a6f6-4a06-a7c6-8fa755d3d08b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:43:24.120458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80342798-a6f6-4a06-a7c6-8fa755d3d08b service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:24.120741 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80342798-a6f6-4a06-a7c6-8fa755d3d08b service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:24.121018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80342798-a6f6-4a06-a7c6-8fa755d3d08b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:24.122652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-80342798-a6f6-4a06-a7c6-8fa755d3d08b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:24.196602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:24.233201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80342798-a6f6-4a06-a7c6-8fa755d3d08b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:43:24.234672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80342798-a6f6-4a06-a7c6-8fa755d3d08b service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:43:24.253957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 744/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:43:26.452567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884dbd5-ae25-4da5-8614-24b90502ed4e None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:26.453152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884dbd5-ae25-4da5-8614-24b90502ed4e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:26.453152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884dbd5-ae25-4da5-8614-24b90502ed4e None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:26.454482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 741/1485] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:43:26.460018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40730c50-3633-48cb-9354-7fd22c8825ab None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:26.460447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40730c50-3633-48cb-9354-7fd22c8825ab None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:26.460568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40730c50-3633-48cb-9354-7fd22c8825ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:26.525750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-40730c50-3633-48cb-9354-7fd22c8825ab None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:43:26.526960 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-40730c50-3633-48cb-9354-7fd22c8825ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:26.620804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 745/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:43:26.920658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884dbd5-ae25-4da5-8614-24b90502ed4e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:43:26.922285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5884dbd5-ae25-4da5-8614-24b90502ed4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:27.025122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884dbd5-ae25-4da5-8614-24b90502ed4e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:43:27.096097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:27.097547 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0639b-19df-4732-8be8-8a970ff313a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:43:27.098220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0639b-19df-4732-8be8-8a970ff313a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:27.098497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0639b-19df-4732-8be8-8a970ff313a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:27.098766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0639b-19df-4732-8be8-8a970ff313a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:27.100327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6bb0639b-19df-4732-8be8-8a970ff313a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:27.210444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0639b-19df-4732-8be8-8a970ff313a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:43:27.211881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0639b-19df-4732-8be8-8a970ff313a7 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:43:27.230118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 742/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:43:29.541691 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179dbe7-5fc0-4990-9529-a8f6e7715f09 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:29.542304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179dbe7-5fc0-4990-9529-a8f6e7715f09 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:29.542304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179dbe7-5fc0-4990-9529-a8f6e7715f09 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:29.543621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 746/1488] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:43:29.549347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2518a-ef69-40cd-b7a0-441d47acf474 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:29.549789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2518a-ef69-40cd-b7a0-441d47acf474 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:29.549903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2518a-ef69-40cd-b7a0-441d47acf474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:29.615149 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-69d2518a-ef69-40cd-b7a0-441d47acf474 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:43:29.616359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-69d2518a-ef69-40cd-b7a0-441d47acf474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:29.668398 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:29.712466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 743/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:43:30.000711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179dbe7-5fc0-4990-9529-a8f6e7715f09 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:43:30.002278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c179dbe7-5fc0-4990-9529-a8f6e7715f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:30.039589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:30.108500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179dbe7-5fc0-4990-9529-a8f6e7715f09 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:43:30.183637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71657b2e-7cc0-4219-91bc-b05b8757fa0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:43:30.184237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71657b2e-7cc0-4219-91bc-b05b8757fa0a service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:30.184511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71657b2e-7cc0-4219-91bc-b05b8757fa0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:30.184781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71657b2e-7cc0-4219-91bc-b05b8757fa0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:30.186385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-71657b2e-7cc0-4219-91bc-b05b8757fa0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:30.267607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:30.295823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71657b2e-7cc0-4219-91bc-b05b8757fa0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:43:30.297258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71657b2e-7cc0-4219-91bc-b05b8757fa0a service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:43:30.316426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 747/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:43:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:43:32.913745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa72ab-0792-4070-b32a-2800a1ed8d89 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:32.914303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa72ab-0792-4070-b32a-2800a1ed8d89 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:32.914303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa72ab-0792-4070-b32a-2800a1ed8d89 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:32.915876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 744/1491] 10.222.0.42 () {54 vars in 927 bytes} [Sat Sep 17 03:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:43:32.921377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e61df7-7c51-480c-af02-f250dd5182bb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:32.921839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e61df7-7c51-480c-af02-f250dd5182bb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:32.921961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e61df7-7c51-480c-af02-f250dd5182bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:32.987941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-55e61df7-7c51-480c-af02-f250dd5182bb None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:43:32.989060 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-55e61df7-7c51-480c-af02-f250dd5182bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:33.085257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 748/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Sat Sep 17 03:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:43:33.374192 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa72ab-0792-4070-b32a-2800a1ed8d89 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:43:33.375720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6eaa72ab-0792-4070-b32a-2800a1ed8d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:33.478089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa72ab-0792-4070-b32a-2800a1ed8d89 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:43:33.552865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b513f-27cb-473e-be4b-5c770e0ce8d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:43:33.553467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b513f-27cb-473e-be4b-5c770e0ce8d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:43:33.553765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b513f-27cb-473e-be4b-5c770e0ce8d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:43:33.554042 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b513f-27cb-473e-be4b-5c770e0ce8d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:43:33.556232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-df2b513f-27cb-473e-be4b-5c770e0ce8d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:43:33.564768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:33.668004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b513f-27cb-473e-be4b-5c770e0ce8d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:43:33.669446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b513f-27cb-473e-be4b-5c770e0ce8d1 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:43:33.687785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 745/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:43:43.083436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=82 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:43:43.685549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=50 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:45:09.294263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485ed15-30a2-4360-af70-373990b19d89 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:09.294869 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485ed15-30a2-4360-af70-373990b19d89 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:09.294869 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485ed15-30a2-4360-af70-373990b19d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:09.370280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c485ed15-30a2-4360-af70-373990b19d89 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:45:09.371494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c485ed15-30a2-4360-af70-373990b19d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:09.469064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 749/1494] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:45:09.477220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c5419-cdd1-4b91-8328-2085c6ea61ff None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:09.477756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c5419-cdd1-4b91-8328-2085c6ea61ff None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:09.477868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c5419-cdd1-4b91-8328-2085c6ea61ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:09.554729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6e2c5419-cdd1-4b91-8328-2085c6ea61ff None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:45:09.555868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6e2c5419-cdd1-4b91-8328-2085c6ea61ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:09.652055 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 746/1495] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:45:09.656730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485ed15-30a2-4360-af70-373990b19d89 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:45:09.658114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c485ed15-30a2-4360-af70-373990b19d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:09.672282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485ed15-30a2-4360-af70-373990b19d89 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:45:09.735983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:45:09.769338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:45:09.769968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:09.770249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:09.770520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:09.771161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:45:09.771465 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:45:09.772474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:45:09.772474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Sep 17 03:45:09.772474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: Sep 17 03:45:09.773360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:45:09.786144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 750/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:45:09.794581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c5419-cdd1-4b91-8328-2085c6ea61ff None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:45:09.803884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c5419-cdd1-4b91-8328-2085c6ea61ff None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:45:09.866979 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:45:09.901270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bba51-0731-4bd3-b64f-79f2ac4197d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:45:09.901911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bba51-0731-4bd3-b64f-79f2ac4197d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:09.902193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bba51-0731-4bd3-b64f-79f2ac4197d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:09.902476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bba51-0731-4bd3-b64f-79f2ac4197d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:09.903483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bba51-0731-4bd3-b64f-79f2ac4197d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:45:09.905587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: WARNING py.warnings [None req-296bba51-0731-4bd3-b64f-79f2ac4197d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:45:09.905587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: warnings.warn(msg) Sep 17 03:45:09.905587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: Sep 17 03:45:09.906826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bba51-0731-4bd3-b64f-79f2ac4197d4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:45:10.077318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 747/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:45:09 2022] POST /identity/v3/projects => generated 379 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:45:10.081471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:45:10.083088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:10.116766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41486c24-c962-4f0f-9762-e14c6bcb0832 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:45:10.212826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59d851-b483-44c9-a1ca-32d1ef746906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:45:10.213958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59d851-b483-44c9-a1ca-32d1ef746906 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:10.214241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59d851-b483-44c9-a1ca-32d1ef746906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:10.214507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59d851-b483-44c9-a1ca-32d1ef746906 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:10.215136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59d851-b483-44c9-a1ca-32d1ef746906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:45:10.216136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-aa59d851-b483-44c9-a1ca-32d1ef746906 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:45:10.216136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Sep 17 03:45:10.216136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: Sep 17 03:45:10.217341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59d851-b483-44c9-a1ca-32d1ef746906 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:45:10.334890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 751/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:45:10 2022] POST /identity/v3/users => generated 290 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:45:10.339897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bba51-0731-4bd3-b64f-79f2ac4197d4 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:45:10.341416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-296bba51-0731-4bd3-b64f-79f2ac4197d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:10.402315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bba51-0731-4bd3-b64f-79f2ac4197d4 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:45:10.485100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:45:10.494866 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c2504-a496-4adb-8604-e2220cb4d0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:45:10.500261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c2504-a496-4adb-8604-e2220cb4d0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:10.500261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c2504-a496-4adb-8604-e2220cb4d0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:10.500261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c2504-a496-4adb-8604-e2220cb4d0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:10.500261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c2504-a496-4adb-8604-e2220cb4d0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:45:10.500261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: WARNING py.warnings [None req-241c2504-a496-4adb-8604-e2220cb4d0d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:45:10.500261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: warnings.warn(msg) Sep 17 03:45:10.500261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: Sep 17 03:45:10.500885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c2504-a496-4adb-8604-e2220cb4d0d7 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:45:10.629897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 748/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:45:10.634126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59d851-b483-44c9-a1ca-32d1ef746906 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:45:10.636499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-aa59d851-b483-44c9-a1ca-32d1ef746906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:10.650508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59d851-b483-44c9-a1ca-32d1ef746906 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:45:10.675712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:45:10.743744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ae5dd-eff8-4efd-96aa-6cf5c499e1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:45:10.744303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ae5dd-eff8-4efd-96aa-6cf5c499e1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:10.744603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ae5dd-eff8-4efd-96aa-6cf5c499e1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:10.744862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ae5dd-eff8-4efd-96aa-6cf5c499e1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:10.745493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ae5dd-eff8-4efd-96aa-6cf5c499e1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:45:10.746543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-560ae5dd-eff8-4efd-96aa-6cf5c499e1f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:45:10.746543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Sep 17 03:45:10.746543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: Sep 17 03:45:10.747436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ae5dd-eff8-4efd-96aa-6cf5c499e1f3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:45:10.761249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 752/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:45:10.765477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c2504-a496-4adb-8604-e2220cb4d0d7 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:45:10.767003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-241c2504-a496-4adb-8604-e2220cb4d0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:10.799960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c2504-a496-4adb-8604-e2220cb4d0d7 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:45:10.893413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e691a5-1034-40f4-92f1-2d3c9f4ea0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:45:10.894084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e691a5-1034-40f4-92f1-2d3c9f4ea0d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:10.894360 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e691a5-1034-40f4-92f1-2d3c9f4ea0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:10.894629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e691a5-1034-40f4-92f1-2d3c9f4ea0d0 admin admin] PATH_INFO: `/v3/projects/d6fad4d84d364d01902e37a000bfd982/users/d7e41339da9b42e1a7c78dabf32a9de5/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:10.917023 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e691a5-1034-40f4-92f1-2d3c9f4ea0d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6fad4d84d364d01902e37a000bfd982, user_id=d7e41339da9b42e1a7c78dabf32a9de5, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:45:10.918072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: WARNING py.warnings [None req-75e691a5-1034-40f4-92f1-2d3c9f4ea0d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:45:10.918072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: warnings.warn(msg) Sep 17 03:45:10.918072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: Sep 17 03:45:10.919997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e691a5-1034-40f4-92f1-2d3c9f4ea0d0 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:45:11.061817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 749/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:45:10 2022] PUT /identity/v3/projects/d6fad4d84d364d01902e37a000bfd982/users/d7e41339da9b42e1a7c78dabf32a9de5/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:45:11.073964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7668c2-bea7-4549-8fd0-771e913626ae None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:11.074254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7668c2-bea7-4549-8fd0-771e913626ae None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:11.074532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7668c2-bea7-4549-8fd0-771e913626ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:11.132588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-be7668c2-bea7-4549-8fd0-771e913626ae None None] MFA Rules not processed for user `d7e41339da9b42e1a7c78dabf32a9de5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:45:11.133823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-be7668c2-bea7-4549-8fd0-771e913626ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:11.213973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 753/1502] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:45:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:45:11.227414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333234df-f7ce-4eec-b140-0b30e3fda649 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:11.227700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333234df-f7ce-4eec-b140-0b30e3fda649 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:11.227978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333234df-f7ce-4eec-b140-0b30e3fda649 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:11.229410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 750/1503] 10.222.0.42 () {54 vars in 964 bytes} [Sat Sep 17 03:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:45:11.234323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bae7c6-0364-4630-ac69-b84d693c0a18 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:11.234614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bae7c6-0364-4630-ac69-b84d693c0a18 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:11.234894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bae7c6-0364-4630-ac69-b84d693c0a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:11.277094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:45:11.327239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-18bae7c6-0364-4630-ac69-b84d693c0a18 None None] MFA Rules not processed for user `727f7207d82640a3a228dfb83814148b`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:45:11.328361 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-18bae7c6-0364-4630-ac69-b84d693c0a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:11.404444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 754/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Sat Sep 17 03:45:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:45:11.412263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333234df-f7ce-4eec-b140-0b30e3fda649 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:45:11.415380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-333234df-f7ce-4eec-b140-0b30e3fda649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:11.473929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:45:11.533413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333234df-f7ce-4eec-b140-0b30e3fda649 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:45:11.537420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782fd19-692f-4a3e-8d88-f041bd207125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:45:11.538056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782fd19-692f-4a3e-8d88-f041bd207125 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:11.538337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782fd19-692f-4a3e-8d88-f041bd207125 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:11.538613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782fd19-692f-4a3e-8d88-f041bd207125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:11.540338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5782fd19-692f-4a3e-8d88-f041bd207125 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:11.637577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5782fd19-692f-4a3e-8d88-f041bd207125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:45:11.638975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5782fd19-692f-4a3e-8d88-f041bd207125 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:45:11.657165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 751/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:45:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:45:11.703261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a00d-bb40-4b3a-b2b6-497afc175f1b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:11.703549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a00d-bb40-4b3a-b2b6-497afc175f1b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:11.703849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a00d-bb40-4b3a-b2b6-497afc175f1b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:11.705099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 755/1506] 10.222.0.42 () {52 vars in 830 bytes} [Sat Sep 17 03:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:45:11.708691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782fd19-692f-4a3e-8d88-f041bd207125 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:45:11.710111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5782fd19-692f-4a3e-8d88-f041bd207125 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:11.782664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:45:11.811657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782fd19-692f-4a3e-8d88-f041bd207125 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:45:11.816210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:45:11.816836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:11.817320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:11.817834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] PATH_INFO: `/v3/users/d7e41339da9b42e1a7c78dabf32a9de5` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:11.819272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7e41339da9b42e1a7c78dabf32a9de5)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:45:11.820294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: WARNING py.warnings [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:45:11.820294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: warnings.warn(msg) Sep 17 03:45:11.820294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: Sep 17 03:45:11.821500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:45:12.070577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7e41339da9b42e1a7c78dabf32a9de5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:45:12.176656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7e41339da9b42e1a7c78dabf32a9de5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:45:12.177278 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7e41339da9b42e1a7c78dabf32a9de5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:45:12.191629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7e41339da9b42e1a7c78dabf32a9de5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:45:12.216096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7e41339da9b42e1a7c78dabf32a9de5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:45:12.222174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 752/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:45:11 2022] DELETE /identity/v3/users/d7e41339da9b42e1a7c78dabf32a9de5 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:45:12.226626 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78661-5132-4dd0-ac5d-9a43bc88bd52 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:12.226912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78661-5132-4dd0-ac5d-9a43bc88bd52 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:12.227188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78661-5132-4dd0-ac5d-9a43bc88bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:12.255042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4ef78661-5132-4dd0-ac5d-9a43bc88bd52 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:45:12.256223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4ef78661-5132-4dd0-ac5d-9a43bc88bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:12.315164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 756/1508] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:45:12.325719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:45:12.327098 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:12.405254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcb366-7564-4960-8dc7-9f1dc9fc10f5 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:45:12.408883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e8e0b-5913-4f7a-9fd6-a92e161ef7e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:45:12.409486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e8e0b-5913-4f7a-9fd6-a92e161ef7e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:12.409786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e8e0b-5913-4f7a-9fd6-a92e161ef7e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:12.410064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e8e0b-5913-4f7a-9fd6-a92e161ef7e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:12.419113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e8e0b-5913-4f7a-9fd6-a92e161ef7e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:45:12.420537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e8e0b-5913-4f7a-9fd6-a92e161ef7e9 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:45:12.429856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 753/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:45:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:45:12.923541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef78661-5132-4dd0-ac5d-9a43bc88bd52 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:45:12.925160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4ef78661-5132-4dd0-ac5d-9a43bc88bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:45:12.940864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef78661-5132-4dd0-ac5d-9a43bc88bd52 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:45:12.946227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:45:12.946868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:45:12.947149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:45:12.947533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] PATH_INFO: `/v3/projects/d6fad4d84d364d01902e37a000bfd982` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:45:12.949017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6fad4d84d364d01902e37a000bfd982)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:45:12.950094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:45:12.950094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Sep 17 03:45:12.950094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: Sep 17 03:45:12.956829 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:45:13.107045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:45:13.110302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6fad4d84d364d01902e37a000bfd982', 'request_id': 'req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:45:13.124614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6fad4d84d364d01902e37a000bfd982', 'request_id': 'req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:45:13.246780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6fad4d84d364d01902e37a000bfd982', 'request_id': 'req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:45:13.258713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] The token cache is being invalidate because project d6fad4d84d364d01902e37a000bfd982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:45:13.265630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:45:13.265630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0bbd17e3-23d4-402b-9e0c-c9dae2f27abe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:45:13.265630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 757/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:45:12 2022] DELETE /identity/v3/projects/d6fad4d84d364d01902e37a000bfd982 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:45:22.425699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=72,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:45:23.446152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:46:40.510633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac4b30-317e-42f5-8748-9ec2571bfc68 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:40.511202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac4b30-317e-42f5-8748-9ec2571bfc68 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:40.511202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac4b30-317e-42f5-8748-9ec2571bfc68 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:40.512436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:46:40.517368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13811c5-2e23-478c-bcb3-9a3dc01018e4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:40.517846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13811c5-2e23-478c-bcb3-9a3dc01018e4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:40.517953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13811c5-2e23-478c-bcb3-9a3dc01018e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:40.566104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d13811c5-2e23-478c-bcb3-9a3dc01018e4 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:46:40.567283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d13811c5-2e23-478c-bcb3-9a3dc01018e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:46:40.646703 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:46:40.800582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac4b30-317e-42f5-8748-9ec2571bfc68 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:46:40.802301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-26ac4b30-317e-42f5-8748-9ec2571bfc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:46:40.905812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac4b30-317e-42f5-8748-9ec2571bfc68 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:46:40.911237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd99b2-e809-499d-a579-fd78d26c0e65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:46:40.911850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd99b2-e809-499d-a579-fd78d26c0e65 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:40.912127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd99b2-e809-499d-a579-fd78d26c0e65 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:40.912400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd99b2-e809-499d-a579-fd78d26c0e65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:40.920763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd99b2-e809-499d-a579-fd78d26c0e65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:46:40.922381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd99b2-e809-499d-a579-fd78d26c0e65 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:46:40.932080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 755/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:46:42.024354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c7e66-5d9b-4c12-8843-88a0b0ba9cb9 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:42.024932 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c7e66-5d9b-4c12-8843-88a0b0ba9cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:42.024932 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c7e66-5d9b-4c12-8843-88a0b0ba9cb9 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:42.026334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:46:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:46:42.032863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cccf5-da14-4197-bdb3-063604bd0fc9 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:42.033334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cccf5-da14-4197-bdb3-063604bd0fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:42.033449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cccf5-da14-4197-bdb3-063604bd0fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:42.053652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d71cccf5-da14-4197-bdb3-063604bd0fc9 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:46:42.055164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d71cccf5-da14-4197-bdb3-063604bd0fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:46:42.063156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:46:42.123468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b307612-0235-4cc6-8475-f8741d891818 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:42.123777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b307612-0235-4cc6-8475-f8741d891818 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:42.124062 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b307612-0235-4cc6-8475-f8741d891818 None None] PATH_INFO: `/v3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:42.125396 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 760/1516] 10.222.0.18 () {54 vars in 925 bytes} [Sat Sep 17 03:46:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:46:42.133014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760a18-d844-46f3-b0ef-7ac007c933c0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:42.133298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760a18-d844-46f3-b0ef-7ac007c933c0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:42.133571 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760a18-d844-46f3-b0ef-7ac007c933c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:42.191421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4b760a18-d844-46f3-b0ef-7ac007c933c0 None None] MFA Rules not processed for user `34ec5c9c71d14762a4869b7d66e87ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:46:42.192611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4b760a18-d844-46f3-b0ef-7ac007c933c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:46:42.221149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b307612-0235-4cc6-8475-f8741d891818 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:46:42.236154 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b307612-0235-4cc6-8475-f8741d891818 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:46:42.241532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0eecef-23a1-4f1e-8a9d-d5e2e76609ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:46:42.242287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eecef-23a1-4f1e-8a9d-d5e2e76609ca service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:42.242551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eecef-23a1-4f1e-8a9d-d5e2e76609ca service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:42.242824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eecef-23a1-4f1e-8a9d-d5e2e76609ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:42.251326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0eecef-23a1-4f1e-8a9d-d5e2e76609ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:46:42.252942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0eecef-23a1-4f1e-8a9d-d5e2e76609ca service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:46:42.263996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 761/1517] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:46:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:46:42.271910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 757/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Sat Sep 17 03:46:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:46:51.201883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca43b-3978-44b9-8cbd-b18e5bdd0f59 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:51.202403 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca43b-3978-44b9-8cbd-b18e5bdd0f59 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:51.202507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca43b-3978-44b9-8cbd-b18e5bdd0f59 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:51.203745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 762/1519] 10.222.0.42 () {54 vars in 961 bytes} [Sat Sep 17 03:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:46:51.208660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea8326-5aad-4cc8-962c-3e7d5c9adcc0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:51.209088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea8326-5aad-4cc8-962c-3e7d5c9adcc0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:51.209217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea8326-5aad-4cc8-962c-3e7d5c9adcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:51.224285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:46:51.230916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-73ea8326-5aad-4cc8-962c-3e7d5c9adcc0 None None] MFA Rules not processed for user `34ec5c9c71d14762a4869b7d66e87ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:46:51.232098 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-73ea8326-5aad-4cc8-962c-3e7d5c9adcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:46:51.240659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 758/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Sat Sep 17 03:46:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:46:51.248882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ca43b-3978-44b9-8cbd-b18e5bdd0f59 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:46:51.261997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ca43b-3978-44b9-8cbd-b18e5bdd0f59 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:46:51.266875 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea816b45-68d6-4991-adc3-87c57a1e0fc2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ec5c9c71d14762a4869b7d66e87ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:46:51.267456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea816b45-68d6-4991-adc3-87c57a1e0fc2 service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:51.267733 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea816b45-68d6-4991-adc3-87c57a1e0fc2 service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:51.268003 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea816b45-68d6-4991-adc3-87c57a1e0fc2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:51.276053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea816b45-68d6-4991-adc3-87c57a1e0fc2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:46:51.278531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea816b45-68d6-4991-adc3-87c57a1e0fc2 service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:46:51.287117 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 763/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:46:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:46:53.785780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148099d1-03c0-402e-ab56-dd979c994aed None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:53.786381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148099d1-03c0-402e-ab56-dd979c994aed None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:53.786381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148099d1-03c0-402e-ab56-dd979c994aed None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:53.787849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:46:53.794678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e63ec-4dfa-4ef3-8550-dcc4e6ec540a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:53.795229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e63ec-4dfa-4ef3-8550-dcc4e6ec540a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:53.795349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e63ec-4dfa-4ef3-8550-dcc4e6ec540a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:53.818335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5f4e63ec-4dfa-4ef3-8550-dcc4e6ec540a None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:46:53.819545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5f4e63ec-4dfa-4ef3-8550-dcc4e6ec540a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:46:53.828650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:46:54.011973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148099d1-03c0-402e-ab56-dd979c994aed None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:46:54.029168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148099d1-03c0-402e-ab56-dd979c994aed None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:46:54.034817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b635fc-4e40-4501-9616-cc65db21a377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:46:54.035411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b635fc-4e40-4501-9616-cc65db21a377 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:54.035684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b635fc-4e40-4501-9616-cc65db21a377 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:54.035999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b635fc-4e40-4501-9616-cc65db21a377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:54.047334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b635fc-4e40-4501-9616-cc65db21a377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:46:54.048934 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b635fc-4e40-4501-9616-cc65db21a377 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:46:54.061905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 760/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:46:55.170342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0424e-3a70-4e97-b64e-21b3f86f8e3e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:55.170890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0424e-3a70-4e97-b64e-21b3f86f8e3e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:55.170890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0424e-3a70-4e97-b64e-21b3f86f8e3e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:55.172239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 03:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:46:55.178301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9e7e4-e4f2-4ecb-a38d-83ee00212ee3 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:55.178742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9e7e4-e4f2-4ecb-a38d-83ee00212ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:55.178854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9e7e4-e4f2-4ecb-a38d-83ee00212ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:55.199183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-c5f9e7e4-e4f2-4ecb-a38d-83ee00212ee3 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:46:55.200390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c5f9e7e4-e4f2-4ecb-a38d-83ee00212ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:46:55.208282 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 03:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:46:55.360306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0424e-3a70-4e97-b64e-21b3f86f8e3e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:46:55.375216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0424e-3a70-4e97-b64e-21b3f86f8e3e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:46:55.380557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b443eed-42c1-45fe-a400-32969aeccc70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:46:55.381137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b443eed-42c1-45fe-a400-32969aeccc70 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:55.381412 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b443eed-42c1-45fe-a400-32969aeccc70 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:55.381681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b443eed-42c1-45fe-a400-32969aeccc70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:55.390678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b443eed-42c1-45fe-a400-32969aeccc70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:46:55.392244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b443eed-42c1-45fe-a400-32969aeccc70 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:46:55.401678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 766/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:46:59.488958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d8a13-6e1c-428f-a402-9171e91ea570 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:59.489525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d8a13-6e1c-428f-a402-9171e91ea570 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:59.489525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d8a13-6e1c-428f-a402-9171e91ea570 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:59.491803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:46:59.498711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a9e38-c9cc-42b0-8113-e79e94289b28 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:59.499244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a9e38-c9cc-42b0-8113-e79e94289b28 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:59.499371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a9e38-c9cc-42b0-8113-e79e94289b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:59.519083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-183a9e38-c9cc-42b0-8113-e79e94289b28 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:46:59.520287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-183a9e38-c9cc-42b0-8113-e79e94289b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:46:59.528458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:46:59.832792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d8a13-6e1c-428f-a402-9171e91ea570 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:46:59.834522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9a0d8a13-6e1c-428f-a402-9171e91ea570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:46:59.893250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d8a13-6e1c-428f-a402-9171e91ea570 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:46:59.896769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e6f65-66c5-4b84-a2c9-af22b8ec200e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:46:59.897367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e6f65-66c5-4b84-a2c9-af22b8ec200e service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:46:59.897643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e6f65-66c5-4b84-a2c9-af22b8ec200e service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:46:59.897944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e6f65-66c5-4b84-a2c9-af22b8ec200e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:46:59.913101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e6f65-66c5-4b84-a2c9-af22b8ec200e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:46:59.914558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e6f65-66c5-4b84-a2c9-af22b8ec200e service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:46:59.924519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 763/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:46:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:47:02.088517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371ca01-e50d-495d-a109-ddcf20e487a9 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:47:02.089019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371ca01-e50d-495d-a109-ddcf20e487a9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:47:02.089019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371ca01-e50d-495d-a109-ddcf20e487a9 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:47:02.090417 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:47:02.095730 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefafc69-7b3e-4c0b-8f67-118b41417532 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:47:02.096119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefafc69-7b3e-4c0b-8f67-118b41417532 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:47:02.096281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefafc69-7b3e-4c0b-8f67-118b41417532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:47:02.116137 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-aefafc69-7b3e-4c0b-8f67-118b41417532 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:47:02.117337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-aefafc69-7b3e-4c0b-8f67-118b41417532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:47:02.125249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:47:02.417693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371ca01-e50d-495d-a109-ddcf20e487a9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:47:02.428608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371ca01-e50d-495d-a109-ddcf20e487a9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:47:02.432513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891b898-601a-4c79-8ba2-e9152623d44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:47:02.433104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891b898-601a-4c79-8ba2-e9152623d44e service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:47:02.433388 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891b898-601a-4c79-8ba2-e9152623d44e service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:47:02.433663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891b898-601a-4c79-8ba2-e9152623d44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:47:02.442578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891b898-601a-4c79-8ba2-e9152623d44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:47:02.444007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891b898-601a-4c79-8ba2-e9152623d44e service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:47:02.454083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 769/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:47:06.196556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e1cb5-2c13-4668-8991-78d8fa59ad06 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:47:06.197121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e1cb5-2c13-4668-8991-78d8fa59ad06 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:47:06.197243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e1cb5-2c13-4668-8991-78d8fa59ad06 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:47:06.198629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:47:06.204365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb782a-0eee-4264-aee8-fab1de39b8eb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:47:06.204873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb782a-0eee-4264-aee8-fab1de39b8eb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:47:06.204969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb782a-0eee-4264-aee8-fab1de39b8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:47:06.212999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=69 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:47:06.224941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-74fb782a-0eee-4264-aee8-fab1de39b8eb None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:47:06.227083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-74fb782a-0eee-4264-aee8-fab1de39b8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:47:06.234612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:47:06.540001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e1cb5-2c13-4668-8991-78d8fa59ad06 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:47:06.542517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-168e1cb5-2c13-4668-8991-78d8fa59ad06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:47:06.599337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e1cb5-2c13-4668-8991-78d8fa59ad06 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:47:06.603544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d7dc7-5cb8-438f-8002-e0beb389c218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:47:06.604140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d7dc7-5cb8-438f-8002-e0beb389c218 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:47:06.604419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d7dc7-5cb8-438f-8002-e0beb389c218 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:47:06.604698 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d7dc7-5cb8-438f-8002-e0beb389c218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:47:06.613619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d7dc7-5cb8-438f-8002-e0beb389c218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:47:06.615088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d7dc7-5cb8-438f-8002-e0beb389c218 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:47:06.624662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 766/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:47:08.809289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ace739-657c-47bf-b16c-5bb4bbf9e103 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:47:08.809766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ace739-657c-47bf-b16c-5bb4bbf9e103 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:47:08.809898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ace739-657c-47bf-b16c-5bb4bbf9e103 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:47:08.811232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 03:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:47:08.816492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e872fc-02bf-4fbb-aa40-1028cff9b6cd None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:47:08.816915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e872fc-02bf-4fbb-aa40-1028cff9b6cd None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:47:08.817026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e872fc-02bf-4fbb-aa40-1028cff9b6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:47:08.836594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-77e872fc-02bf-4fbb-aa40-1028cff9b6cd None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:47:08.838314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-77e872fc-02bf-4fbb-aa40-1028cff9b6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:47:08.846207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 03:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:47:09.154164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ace739-657c-47bf-b16c-5bb4bbf9e103 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:47:09.155821 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f1ace739-657c-47bf-b16c-5bb4bbf9e103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:47:09.168997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ace739-657c-47bf-b16c-5bb4bbf9e103 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:47:09.172470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a52139-ff3e-4865-b2c0-2373e7ad6e23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:47:09.173051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a52139-ff3e-4865-b2c0-2373e7ad6e23 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:47:09.173325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a52139-ff3e-4865-b2c0-2373e7ad6e23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:47:09.173591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a52139-ff3e-4865-b2c0-2373e7ad6e23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:47:09.183148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a52139-ff3e-4865-b2c0-2373e7ad6e23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:47:09.184576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a52139-ff3e-4865-b2c0-2373e7ad6e23 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:47:09.193782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 772/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:47:18.830944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=67 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:47:19.189643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=8 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:34.809788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0244cc-dea3-4484-b13d-5e8126fb24c5 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:34.809788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0244cc-dea3-4484-b13d-5e8126fb24c5 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:34.810406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0244cc-dea3-4484-b13d-5e8126fb24c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:34.829192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6d6a4-4009-45ed-af66-b09c1eaced00 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:34.830529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6d6a4-4009-45ed-af66-b09c1eaced00 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:34.831222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6d6a4-4009-45ed-af66-b09c1eaced00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:34.854790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-0c0244cc-dea3-4484-b13d-5e8126fb24c5 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:34.856072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0c0244cc-dea3-4484-b13d-5e8126fb24c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:34.882259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 768/1540] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:34.882652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e5c6d6a4-4009-45ed-af66-b09c1eaced00 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:34.883879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e5c6d6a4-4009-45ed-af66-b09c1eaced00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:34.896338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345a4b8-d7b5-46e0-9cf1-f4b8080282e9 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:34.896625 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345a4b8-d7b5-46e0-9cf1-f4b8080282e9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:34.896905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345a4b8-d7b5-46e0-9cf1-f4b8080282e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:34.904535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 773/1541] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:34.905818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185143ec-be1a-4c1f-abf9-5b9800ab5edb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:34.906910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185143ec-be1a-4c1f-abf9-5b9800ab5edb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:34.907198 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185143ec-be1a-4c1f-abf9-5b9800ab5edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:34.931031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b345a4b8-d7b5-46e0-9cf1-f4b8080282e9 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:34.932442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b345a4b8-d7b5-46e0-9cf1-f4b8080282e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:34.951457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 769/1542] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:34.951898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-185143ec-be1a-4c1f-abf9-5b9800ab5edb None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:34.953664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-185143ec-be1a-4c1f-abf9-5b9800ab5edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:34.955247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589f903-3cc4-4736-8066-20bade8fd5f0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:34.955863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589f903-3cc4-4736-8066-20bade8fd5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:34.957600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589f903-3cc4-4736-8066-20bade8fd5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:34.966508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 774/1543] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:34.967767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313462ab-418e-414f-be89-0b527baab020 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:34.968050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313462ab-418e-414f-be89-0b527baab020 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:34.968325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313462ab-418e-414f-be89-0b527baab020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.004346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-1589f903-3cc4-4736-8066-20bade8fd5f0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.005720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1589f903-3cc4-4736-8066-20bade8fd5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.006516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-313462ab-418e-414f-be89-0b527baab020 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.007811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-313462ab-418e-414f-be89-0b527baab020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.014363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 770/1544] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.016424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce077f98-8b5d-4a22-812d-60aeff667bd7 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.017078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce077f98-8b5d-4a22-812d-60aeff667bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.017078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce077f98-8b5d-4a22-812d-60aeff667bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.020963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 775/1545] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.021915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313462ab-418e-414f-be89-0b527baab020 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.040264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313462ab-418e-414f-be89-0b527baab020 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.047468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ce077f98-8b5d-4a22-812d-60aeff667bd7 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.047468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ce077f98-8b5d-4a22-812d-60aeff667bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.047700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425e739-daa2-4403-8a9e-1eb8fc009c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.047700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e739-daa2-4403-8a9e-1eb8fc009c98 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.047700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e739-daa2-4403-8a9e-1eb8fc009c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.047700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e739-daa2-4403-8a9e-1eb8fc009c98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.048023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425e739-daa2-4403-8a9e-1eb8fc009c98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:48:35.048023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425e739-daa2-4403-8a9e-1eb8fc009c98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.049276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425e739-daa2-4403-8a9e-1eb8fc009c98 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.056090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 771/1546] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.057239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d742-6f7f-4385-a52c-3a90069bf8e9 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.057521 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d742-6f7f-4385-a52c-3a90069bf8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.057813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d742-6f7f-4385-a52c-3a90069bf8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.063837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 776/1547] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:48:35.065035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb7876-8591-40a5-8cc8-50f563a31de7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.065035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb7876-8591-40a5-8cc8-50f563a31de7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.065265 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb7876-8591-40a5-8cc8-50f563a31de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.080052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a955d742-6f7f-4385-a52c-3a90069bf8e9 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.085835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a955d742-6f7f-4385-a52c-3a90069bf8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.085972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9fdb7876-8591-40a5-8cc8-50f563a31de7 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.086237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9fdb7876-8591-40a5-8cc8-50f563a31de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.089655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 772/1548] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.090318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955d742-6f7f-4385-a52c-3a90069bf8e9 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.098432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 777/1549] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.099618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643f19e-4087-4396-8d32-767a2518ac39 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.099897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643f19e-4087-4396-8d32-767a2518ac39 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.100186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643f19e-4087-4396-8d32-767a2518ac39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.106040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955d742-6f7f-4385-a52c-3a90069bf8e9 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.111139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a6531-602c-4810-979c-a08a03a21361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.111733 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a6531-602c-4810-979c-a08a03a21361 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.112007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a6531-602c-4810-979c-a08a03a21361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.112273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a6531-602c-4810-979c-a08a03a21361 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.112908 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a6531-602c-4810-979c-a08a03a21361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:48:35.113207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a6531-602c-4810-979c-a08a03a21361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.114276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: WARNING py.warnings [None req-4e3a6531-602c-4810-979c-a08a03a21361 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:48:35.114276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: warnings.warn(msg) Sep 17 03:48:35.114276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: Sep 17 03:48:35.115160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a6531-602c-4810-979c-a08a03a21361 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.122342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b643f19e-4087-4396-8d32-767a2518ac39 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.123553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b643f19e-4087-4396-8d32-767a2518ac39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.127562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 773/1550] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:48:35.128808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68debe-b483-4b3c-9a77-7df6f10b6bb2 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.129093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68debe-b483-4b3c-9a77-7df6f10b6bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.129365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68debe-b483-4b3c-9a77-7df6f10b6bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.133829 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 778/1551] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.133829 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bb08c-2f88-4d1f-afb8-a65106c27243 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.134092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bb08c-2f88-4d1f-afb8-a65106c27243 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.134339 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bb08c-2f88-4d1f-afb8-a65106c27243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.152007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-9c68debe-b483-4b3c-9a77-7df6f10b6bb2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.153197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9c68debe-b483-4b3c-9a77-7df6f10b6bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.154595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a57bb08c-2f88-4d1f-afb8-a65106c27243 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.155799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a57bb08c-2f88-4d1f-afb8-a65106c27243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.161873 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 774/1552] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.163258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3534efd-4c69-4a54-a825-f4d164d97da4 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.163701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3534efd-4c69-4a54-a825-f4d164d97da4 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.163827 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 779/1553] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.163920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3534efd-4c69-4a54-a825-f4d164d97da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.164306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57bb08c-2f88-4d1f-afb8-a65106c27243 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.173977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57bb08c-2f88-4d1f-afb8-a65106c27243 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.178631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679300cc-51c5-4dd7-825b-76e2501d865b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.179220 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679300cc-51c5-4dd7-825b-76e2501d865b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.179503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679300cc-51c5-4dd7-825b-76e2501d865b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.179772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679300cc-51c5-4dd7-825b-76e2501d865b admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.180393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679300cc-51c5-4dd7-825b-76e2501d865b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.181395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-679300cc-51c5-4dd7-825b-76e2501d865b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:48:35.181395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Sep 17 03:48:35.181395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: Sep 17 03:48:35.182677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679300cc-51c5-4dd7-825b-76e2501d865b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.188357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e3534efd-4c69-4a54-a825-f4d164d97da4 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.189552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e3534efd-4c69-4a54-a825-f4d164d97da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.197596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 775/1554] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.198403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3534efd-4c69-4a54-a825-f4d164d97da4 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.212114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3534efd-4c69-4a54-a825-f4d164d97da4 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.216698 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28af89-cce3-48f3-a9d6-141cca87af84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.217298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28af89-cce3-48f3-a9d6-141cca87af84 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.217572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28af89-cce3-48f3-a9d6-141cca87af84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.217906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28af89-cce3-48f3-a9d6-141cca87af84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.218786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e28af89-cce3-48f3-a9d6-141cca87af84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:48:35.223064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e28af89-cce3-48f3-a9d6-141cca87af84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.223064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e28af89-cce3-48f3-a9d6-141cca87af84 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.232525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 776/1555] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:48:35.233913 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b6866-5136-41aa-974e-c7c5e0e02c91 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.234405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b6866-5136-41aa-974e-c7c5e0e02c91 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.235218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b6866-5136-41aa-974e-c7c5e0e02c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.254914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ae2b6866-5136-41aa-974e-c7c5e0e02c91 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.256107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ae2b6866-5136-41aa-974e-c7c5e0e02c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.263808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 777/1556] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.264693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b6866-5136-41aa-974e-c7c5e0e02c91 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.274985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b6866-5136-41aa-974e-c7c5e0e02c91 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.309123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9444a69-c35a-4459-8d73-491ec4423233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.309123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9444a69-c35a-4459-8d73-491ec4423233 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.309123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9444a69-c35a-4459-8d73-491ec4423233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.309123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9444a69-c35a-4459-8d73-491ec4423233 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.309549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9444a69-c35a-4459-8d73-491ec4423233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:48:35.309549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9444a69-c35a-4459-8d73-491ec4423233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.309549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9444a69-c35a-4459-8d73-491ec4423233 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.309549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 778/1557] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:48:35.309549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9444a69-c35a-4459-8d73-491ec4423233 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.309549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9444a69-c35a-4459-8d73-491ec4423233 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.310822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d5251-d314-4306-8210-9682386fde82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.311374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d5251-d314-4306-8210-9682386fde82 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.311649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d5251-d314-4306-8210-9682386fde82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.311917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d5251-d314-4306-8210-9682386fde82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.313560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d5251-d314-4306-8210-9682386fde82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.315557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d5251-d314-4306-8210-9682386fde82 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.382570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 780/1558] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/projects => generated 367 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:35.382824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679300cc-51c5-4dd7-825b-76e2501d865b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.446705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 779/1559] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/projects => generated 389 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:35.446930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679300cc-51c5-4dd7-825b-76e2501d865b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.447041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dff4f-eeec-49cd-aab0-f3a81310baa1 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.447041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dff4f-eeec-49cd-aab0-f3a81310baa1 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.447041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dff4f-eeec-49cd-aab0-f3a81310baa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.465905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-645dff4f-eeec-49cd-aab0-f3a81310baa1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.467093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-645dff4f-eeec-49cd-aab0-f3a81310baa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.504870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62789232-edba-4c00-83f5-dc1e872741b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.505645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62789232-edba-4c00-83f5-dc1e872741b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.505975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62789232-edba-4c00-83f5-dc1e872741b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.506280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62789232-edba-4c00-83f5-dc1e872741b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.506928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62789232-edba-4c00-83f5-dc1e872741b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:48:35.507236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62789232-edba-4c00-83f5-dc1e872741b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.508901 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62789232-edba-4c00-83f5-dc1e872741b6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.520782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 781/1560] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:48:35.522236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd07f5-09da-405a-8935-d0c8fa35e024 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.522518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd07f5-09da-405a-8935-d0c8fa35e024 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.522796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd07f5-09da-405a-8935-d0c8fa35e024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.530832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 780/1561] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.532005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dff4f-eeec-49cd-aab0-f3a81310baa1 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.541283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dff4f-eeec-49cd-aab0-f3a81310baa1 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.542571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b6dd07f5-09da-405a-8935-d0c8fa35e024 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:35.543766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b6dd07f5-09da-405a-8935-d0c8fa35e024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:35.545845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfcaca-1397-4cd8-8a99-390e9d601f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.546433 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfcaca-1397-4cd8-8a99-390e9d601f3c admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.547275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfcaca-1397-4cd8-8a99-390e9d601f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.547752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfcaca-1397-4cd8-8a99-390e9d601f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.548594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfcaca-1397-4cd8-8a99-390e9d601f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.550560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfcaca-1397-4cd8-8a99-390e9d601f3c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.552437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 782/1562] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:35.553203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd07f5-09da-405a-8935-d0c8fa35e024 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.562391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd07f5-09da-405a-8935-d0c8fa35e024 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.567557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275afc2f-d190-470b-b657-77284ed8cca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.568145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275afc2f-d190-470b-b657-77284ed8cca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.568422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275afc2f-d190-470b-b657-77284ed8cca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.568700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275afc2f-d190-470b-b657-77284ed8cca0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.569336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275afc2f-d190-470b-b657-77284ed8cca0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:48:35.569641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275afc2f-d190-470b-b657-77284ed8cca0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.571275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275afc2f-d190-470b-b657-77284ed8cca0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.578055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:35.587392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 783/1563] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:48:35.587392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275afc2f-d190-470b-b657-77284ed8cca0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.593529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275afc2f-d190-470b-b657-77284ed8cca0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.598301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062faf0-c08d-4310-ab06-dbdc64868b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.598868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062faf0-c08d-4310-ab06-dbdc64868b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.599151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062faf0-c08d-4310-ab06-dbdc64868b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.599424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062faf0-c08d-4310-ab06-dbdc64868b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.600039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062faf0-c08d-4310-ab06-dbdc64868b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.601918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062faf0-c08d-4310-ab06-dbdc64868b4c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.659331 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 781/1564] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:35.660153 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfcaca-1397-4cd8-8a99-390e9d601f3c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.686794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:35.721361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfcaca-1397-4cd8-8a99-390e9d601f3c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.725960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552cdc4-3360-4822-81ca-f47962cff30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.726552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552cdc4-3360-4822-81ca-f47962cff30b admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.726826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552cdc4-3360-4822-81ca-f47962cff30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.727094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552cdc4-3360-4822-81ca-f47962cff30b admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.727700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552cdc4-3360-4822-81ca-f47962cff30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.728707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: WARNING py.warnings [None req-2552cdc4-3360-4822-81ca-f47962cff30b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:48:35.728707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: warnings.warn(msg) Sep 17 03:48:35.728707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: Sep 17 03:48:35.729916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552cdc4-3360-4822-81ca-f47962cff30b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.735432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 784/1565] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:35.736439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062faf0-c08d-4310-ab06-dbdc64868b4c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.797393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062faf0-c08d-4310-ab06-dbdc64868b4c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.802673 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c67db6-3fd0-459d-ae84-ee77ccc54982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.803131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c67db6-3fd0-459d-ae84-ee77ccc54982 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.803417 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c67db6-3fd0-459d-ae84-ee77ccc54982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.803693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c67db6-3fd0-459d-ae84-ee77ccc54982 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.804328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c67db6-3fd0-459d-ae84-ee77ccc54982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.806337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c67db6-3fd0-459d-ae84-ee77ccc54982 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.853514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 782/1566] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/users => generated 284 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:35.854413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552cdc4-3360-4822-81ca-f47962cff30b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.866227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552cdc4-3360-4822-81ca-f47962cff30b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.870848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ed9a4-4ecf-42ac-96f4-e39bc3644c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.871426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ed9a4-4ecf-42ac-96f4-e39bc3644c9c admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.871728 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ed9a4-4ecf-42ac-96f4-e39bc3644c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.872000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ed9a4-4ecf-42ac-96f4-e39bc3644c9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.872616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ed9a4-4ecf-42ac-96f4-e39bc3644c9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.874554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ed9a4-4ecf-42ac-96f4-e39bc3644c9c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:35.979138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 785/1567] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/users => generated 295 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:35.979980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c67db6-3fd0-459d-ae84-ee77ccc54982 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:35.990448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c67db6-3fd0-459d-ae84-ee77ccc54982 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:35.995055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d1e72-4b60-4df1-a15d-704867bd457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:35.995627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d1e72-4b60-4df1-a15d-704867bd457e admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:35.995911 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d1e72-4b60-4df1-a15d-704867bd457e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:35.996188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d1e72-4b60-4df1-a15d-704867bd457e admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:35.996825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d1e72-4b60-4df1-a15d-704867bd457e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:48:35.997130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d1e72-4b60-4df1-a15d-704867bd457e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:35.998756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d1e72-4b60-4df1-a15d-704867bd457e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.024678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 786/1568] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:48:36.024678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d1e72-4b60-4df1-a15d-704867bd457e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.024678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d1e72-4b60-4df1-a15d-704867bd457e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.025915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15bb76-cda6-46ce-a9e0-e666851042c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.026489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15bb76-cda6-46ce-a9e0-e666851042c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.026771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15bb76-cda6-46ce-a9e0-e666851042c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.027046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15bb76-cda6-46ce-a9e0-e666851042c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.027677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15bb76-cda6-46ce-a9e0-e666851042c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:48:36.027982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15bb76-cda6-46ce-a9e0-e666851042c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.029538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15bb76-cda6-46ce-a9e0-e666851042c6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.041710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 787/1569] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:48:36.042506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15bb76-cda6-46ce-a9e0-e666851042c6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.051104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 783/1570] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:35 2022] POST /identity/v3/projects => generated 379 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:36.053082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ed9a4-4ecf-42ac-96f4-e39bc3644c9c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.059974 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15bb76-cda6-46ce-a9e0-e666851042c6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.115664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ed9a4-4ecf-42ac-96f4-e39bc3644c9c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.117866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e5fc2-dd7d-407d-9328-6a7830a597e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.118476 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e5fc2-dd7d-407d-9328-6a7830a597e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.118754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e5fc2-dd7d-407d-9328-6a7830a597e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.119032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e5fc2-dd7d-407d-9328-6a7830a597e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.119659 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e5fc2-dd7d-407d-9328-6a7830a597e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.120600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f614d3-24d9-42da-a7b2-09a91e076fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.121322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f614d3-24d9-42da-a7b2-09a91e076fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.121597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e5fc2-dd7d-407d-9328-6a7830a597e6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.121706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f614d3-24d9-42da-a7b2-09a91e076fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.127532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f614d3-24d9-42da-a7b2-09a91e076fe3 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.127532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f614d3-24d9-42da-a7b2-09a91e076fe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.127532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f614d3-24d9-42da-a7b2-09a91e076fe3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.253604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 784/1571] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:36 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:36.254456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f614d3-24d9-42da-a7b2-09a91e076fe3 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.263798 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f614d3-24d9-42da-a7b2-09a91e076fe3 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.266774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 788/1572] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:36 2022] POST /identity/v3/projects => generated 375 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:36.267890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e5fc2-dd7d-407d-9328-6a7830a597e6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.305198 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:36.327563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68456db6-4499-4cf6-a7db-576b940b99ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.328709 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68456db6-4499-4cf6-a7db-576b940b99ee admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.328988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68456db6-4499-4cf6-a7db-576b940b99ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.329259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68456db6-4499-4cf6-a7db-576b940b99ee admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.329920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68456db6-4499-4cf6-a7db-576b940b99ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.331915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68456db6-4499-4cf6-a7db-576b940b99ee admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.345280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e5fc2-dd7d-407d-9328-6a7830a597e6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.350001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129458c-dd50-445f-9ed5-fafe8e4bb8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.350711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129458c-dd50-445f-9ed5-fafe8e4bb8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.350996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129458c-dd50-445f-9ed5-fafe8e4bb8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.351271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129458c-dd50-445f-9ed5-fafe8e4bb8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.351910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129458c-dd50-445f-9ed5-fafe8e4bb8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.353501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129458c-dd50-445f-9ed5-fafe8e4bb8f7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.363520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 789/1573] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:36.364303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129458c-dd50-445f-9ed5-fafe8e4bb8f7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.381543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129458c-dd50-445f-9ed5-fafe8e4bb8f7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.381543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d4d29-5e49-45fb-93a4-bb5ce221a6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.381543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d4d29-5e49-45fb-93a4-bb5ce221a6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.381543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d4d29-5e49-45fb-93a4-bb5ce221a6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.381988 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d4d29-5e49-45fb-93a4-bb5ce221a6f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.381988 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d4d29-5e49-45fb-93a4-bb5ce221a6f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.381988 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d4d29-5e49-45fb-93a4-bb5ce221a6f1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.392581 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 790/1574] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:36.393605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d4d29-5e49-45fb-93a4-bb5ce221a6f1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.402861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d4d29-5e49-45fb-93a4-bb5ce221a6f1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.408158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e28bf9-b87f-44d7-8fe7-60ef3b9bd303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.408729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e28bf9-b87f-44d7-8fe7-60ef3b9bd303 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.410258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e28bf9-b87f-44d7-8fe7-60ef3b9bd303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.410258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e28bf9-b87f-44d7-8fe7-60ef3b9bd303 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.410258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e28bf9-b87f-44d7-8fe7-60ef3b9bd303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.411832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e28bf9-b87f-44d7-8fe7-60ef3b9bd303 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.464781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 785/1575] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:36 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:36.465593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68456db6-4499-4cf6-a7db-576b940b99ee admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.474971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68456db6-4499-4cf6-a7db-576b940b99ee admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.479475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f124a56-f89d-4a28-b136-1013af2f1849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.480038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f124a56-f89d-4a28-b136-1013af2f1849 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.480311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f124a56-f89d-4a28-b136-1013af2f1849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.480578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f124a56-f89d-4a28-b136-1013af2f1849 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.481182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f124a56-f89d-4a28-b136-1013af2f1849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.483128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f124a56-f89d-4a28-b136-1013af2f1849 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.492975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:36.535441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 791/1576] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:36 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:36.536294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e28bf9-b87f-44d7-8fe7-60ef3b9bd303 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.586383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 786/1577] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:36 2022] POST /identity/v3/projects => generated 391 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:36.587450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f124a56-f89d-4a28-b136-1013af2f1849 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.601307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e28bf9-b87f-44d7-8fe7-60ef3b9bd303 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.649702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f124a56-f89d-4a28-b136-1013af2f1849 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.654310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d6989-0193-4cb0-95cd-6dc2612d1a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.654910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d6989-0193-4cb0-95cd-6dc2612d1a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.655223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d6989-0193-4cb0-95cd-6dc2612d1a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.655451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d6989-0193-4cb0-95cd-6dc2612d1a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.656628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d6989-0193-4cb0-95cd-6dc2612d1a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.657482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87e146-13d1-4510-a625-196d2a6772e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.658119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87e146-13d1-4510-a625-196d2a6772e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.658395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87e146-13d1-4510-a625-196d2a6772e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.658633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d6989-0193-4cb0-95cd-6dc2612d1a10 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.658730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87e146-13d1-4510-a625-196d2a6772e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.659268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87e146-13d1-4510-a625-196d2a6772e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.661199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87e146-13d1-4510-a625-196d2a6772e2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.668452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 787/1578] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:36.669229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d6989-0193-4cb0-95cd-6dc2612d1a10 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.680702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d6989-0193-4cb0-95cd-6dc2612d1a10 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.685263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b4ec2-6ec4-4536-a375-73d7057898fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.685867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b4ec2-6ec4-4536-a375-73d7057898fa admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.686143 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b4ec2-6ec4-4536-a375-73d7057898fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.686414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b4ec2-6ec4-4536-a375-73d7057898fa admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.687023 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b4ec2-6ec4-4536-a375-73d7057898fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.689242 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b4ec2-6ec4-4536-a375-73d7057898fa admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.761765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 792/1579] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:36 2022] POST /identity/v3/users => generated 290 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:36.762752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87e146-13d1-4510-a625-196d2a6772e2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.772178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87e146-13d1-4510-a625-196d2a6772e2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.776809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6ba82-dd63-4755-a672-c392cab93284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.777379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6ba82-dd63-4755-a672-c392cab93284 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.777661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6ba82-dd63-4755-a672-c392cab93284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.777967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6ba82-dd63-4755-a672-c392cab93284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.778591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6ba82-dd63-4755-a672-c392cab93284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.780168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6ba82-dd63-4755-a672-c392cab93284 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.790474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 793/1580] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:36.790632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 788/1581] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:36 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:36.791395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6ba82-dd63-4755-a672-c392cab93284 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.794538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b4ec2-6ec4-4536-a375-73d7057898fa admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.802345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6ba82-dd63-4755-a672-c392cab93284 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.805541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b4ec2-6ec4-4536-a375-73d7057898fa admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.807906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adddefd-f7bd-47d2-9a96-cdb75db845f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.808505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adddefd-f7bd-47d2-9a96-cdb75db845f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.809658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adddefd-f7bd-47d2-9a96-cdb75db845f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.809658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adddefd-f7bd-47d2-9a96-cdb75db845f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.809982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adddefd-f7bd-47d2-9a96-cdb75db845f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.810268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db72d7-72fe-4f4d-b88c-8926899e6917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.810860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db72d7-72fe-4f4d-b88c-8926899e6917 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.811131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db72d7-72fe-4f4d-b88c-8926899e6917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.811334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adddefd-f7bd-47d2-9a96-cdb75db845f7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.811464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db72d7-72fe-4f4d-b88c-8926899e6917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.812013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db72d7-72fe-4f4d-b88c-8926899e6917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.813633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db72d7-72fe-4f4d-b88c-8926899e6917 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.822315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 794/1582] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:36.823235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adddefd-f7bd-47d2-9a96-cdb75db845f7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.824354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 789/1583] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:36.825126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db72d7-72fe-4f4d-b88c-8926899e6917 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.832389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:36.834993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db72d7-72fe-4f4d-b88c-8926899e6917 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.838020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adddefd-f7bd-47d2-9a96-cdb75db845f7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.840105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2404df-7d82-4898-83f5-08f38aad455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.840761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2404df-7d82-4898-83f5-08f38aad455b admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.841037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2404df-7d82-4898-83f5-08f38aad455b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.841310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2404df-7d82-4898-83f5-08f38aad455b admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.841995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2404df-7d82-4898-83f5-08f38aad455b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.842647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a458e-898c-4413-bb11-bb73f2c164fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.844122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a458e-898c-4413-bb11-bb73f2c164fa admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.844122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a458e-898c-4413-bb11-bb73f2c164fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.844373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a458e-898c-4413-bb11-bb73f2c164fa admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.844843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2404df-7d82-4898-83f5-08f38aad455b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.844978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a458e-898c-4413-bb11-bb73f2c164fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.846870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a458e-898c-4413-bb11-bb73f2c164fa admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.967673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 790/1584] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:36 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:36.969160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2404df-7d82-4898-83f5-08f38aad455b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.979469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2404df-7d82-4898-83f5-08f38aad455b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.982735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 795/1585] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:36 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:36.982735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a458e-898c-4413-bb11-bb73f2c164fa admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.984455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97befe-f857-41ec-a56f-ddd5a4d5756b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.985046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97befe-f857-41ec-a56f-ddd5a4d5756b admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.985319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97befe-f857-41ec-a56f-ddd5a4d5756b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.985584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97befe-f857-41ec-a56f-ddd5a4d5756b admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.986271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97befe-f857-41ec-a56f-ddd5a4d5756b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.987852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97befe-f857-41ec-a56f-ddd5a4d5756b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:36.991052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a458e-898c-4413-bb11-bb73f2c164fa admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:36.995783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02a5cf-0f11-42d5-96a6-8274ba004ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:36.996373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02a5cf-0f11-42d5-96a6-8274ba004ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:36.996654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02a5cf-0f11-42d5-96a6-8274ba004ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:36.996934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02a5cf-0f11-42d5-96a6-8274ba004ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:36.997572 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02a5cf-0f11-42d5-96a6-8274ba004ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:36.998103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 791/1586] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:36.998721 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97befe-f857-41ec-a56f-ddd5a4d5756b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:36.999203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02a5cf-0f11-42d5-96a6-8274ba004ba6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.009330 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97befe-f857-41ec-a56f-ddd5a4d5756b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.011026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 796/1587] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:37.011982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02a5cf-0f11-42d5-96a6-8274ba004ba6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.014229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b850e-a6c2-4220-902c-7e884b6fafd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.014872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b850e-a6c2-4220-902c-7e884b6fafd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.015150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b850e-a6c2-4220-902c-7e884b6fafd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.016038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b850e-a6c2-4220-902c-7e884b6fafd1 admin admin] PATH_INFO: `/v3/projects/3b73849a79164336aa88fbcda196c161/users/b038a963f86849a5b755b24f4bf202f5/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.022112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02a5cf-0f11-42d5-96a6-8274ba004ba6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.026656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1ff2f-e8c3-4339-9f56-167e195dacd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.027250 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1ff2f-e8c3-4339-9f56-167e195dacd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.027533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1ff2f-e8c3-4339-9f56-167e195dacd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.027806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1ff2f-e8c3-4339-9f56-167e195dacd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.028436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1ff2f-e8c3-4339-9f56-167e195dacd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.030050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1ff2f-e8c3-4339-9f56-167e195dacd1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.031035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b850e-a6c2-4220-902c-7e884b6fafd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b73849a79164336aa88fbcda196c161, user_id=b038a963f86849a5b755b24f4bf202f5, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.034587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b850e-a6c2-4220-902c-7e884b6fafd1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.042671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 797/1588] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:37.042671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1ff2f-e8c3-4339-9f56-167e195dacd1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.053457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1ff2f-e8c3-4339-9f56-167e195dacd1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.058203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6df70-ea75-4a90-92d2-ebb6277be73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.058824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6df70-ea75-4a90-92d2-ebb6277be73e admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.059101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6df70-ea75-4a90-92d2-ebb6277be73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.059373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6df70-ea75-4a90-92d2-ebb6277be73e admin admin] PATH_INFO: `/v3/projects/c1a07eadb5ea4914a6e1e6f3844afe94/users/2723e5c74bde49f485099b615b36997a/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.073911 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6df70-ea75-4a90-92d2-ebb6277be73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a07eadb5ea4914a6e1e6f3844afe94, user_id=2723e5c74bde49f485099b615b36997a, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.076333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-92f6df70-ea75-4a90-92d2-ebb6277be73e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:48:37.076333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Sep 17 03:48:37.076333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: Sep 17 03:48:37.076950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6df70-ea75-4a90-92d2-ebb6277be73e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.104510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 792/1589] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:36 2022] PUT /identity/v3/projects/3b73849a79164336aa88fbcda196c161/users/b038a963f86849a5b755b24f4bf202f5/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:37.105343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b850e-a6c2-4220-902c-7e884b6fafd1 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.150777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 798/1590] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:37 2022] PUT /identity/v3/projects/c1a07eadb5ea4914a6e1e6f3844afe94/users/2723e5c74bde49f485099b615b36997a/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:37.151811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6df70-ea75-4a90-92d2-ebb6277be73e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.167949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b850e-a6c2-4220-902c-7e884b6fafd1 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.176365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:37.215978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6df70-ea75-4a90-92d2-ebb6277be73e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.221981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4f8f0-8da3-412b-a675-1d80c1f62e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.222540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4f8f0-8da3-412b-a675-1d80c1f62e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.222817 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4f8f0-8da3-412b-a675-1d80c1f62e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.223088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4f8f0-8da3-412b-a675-1d80c1f62e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.223730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4f8f0-8da3-412b-a675-1d80c1f62e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.225316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4f8f0-8da3-412b-a675-1d80c1f62e8a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.226635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506bfaa-21e0-4d02-bf5f-01bab5bcda2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.227519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506bfaa-21e0-4d02-bf5f-01bab5bcda2d admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.227519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506bfaa-21e0-4d02-bf5f-01bab5bcda2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.227756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506bfaa-21e0-4d02-bf5f-01bab5bcda2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.228380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506bfaa-21e0-4d02-bf5f-01bab5bcda2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.230037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506bfaa-21e0-4d02-bf5f-01bab5bcda2d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.259941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 799/1591] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:37.259941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4f8f0-8da3-412b-a675-1d80c1f62e8a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.260203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 793/1592] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:37.260203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506bfaa-21e0-4d02-bf5f-01bab5bcda2d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.260376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4f8f0-8da3-412b-a675-1d80c1f62e8a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.260467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506bfaa-21e0-4d02-bf5f-01bab5bcda2d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.260570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d0318-8aae-45a6-a5b9-c9383ec6efeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.260570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d0318-8aae-45a6-a5b9-c9383ec6efeb admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.260570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d0318-8aae-45a6-a5b9-c9383ec6efeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.260570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d0318-8aae-45a6-a5b9-c9383ec6efeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.260919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d0318-8aae-45a6-a5b9-c9383ec6efeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.260919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d0318-8aae-45a6-a5b9-c9383ec6efeb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.261095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724618d-8234-4e28-b094-02d7777c7295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.261095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724618d-8234-4e28-b094-02d7777c7295 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.261095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724618d-8234-4e28-b094-02d7777c7295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.261095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724618d-8234-4e28-b094-02d7777c7295 admin admin] PATH_INFO: `/v3/projects/fb6cb03abf5242b594efcf325909dd32/users/fafa609d94334db6907b401e66cb3448/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.267392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 800/1593] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:37.268282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d0318-8aae-45a6-a5b9-c9383ec6efeb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.273428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724618d-8234-4e28-b094-02d7777c7295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6cb03abf5242b594efcf325909dd32, user_id=fafa609d94334db6907b401e66cb3448, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.276265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724618d-8234-4e28-b094-02d7777c7295 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.277821 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d0318-8aae-45a6-a5b9-c9383ec6efeb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.283971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf25557-85fb-41b3-971f-75057565e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.284561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf25557-85fb-41b3-971f-75057565e861 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.284848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf25557-85fb-41b3-971f-75057565e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.285128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf25557-85fb-41b3-971f-75057565e861 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.285826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf25557-85fb-41b3-971f-75057565e861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.287468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf25557-85fb-41b3-971f-75057565e861 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.298438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 801/1594] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:37.300099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf25557-85fb-41b3-971f-75057565e861 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.308545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf25557-85fb-41b3-971f-75057565e861 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.313417 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7314547-b8ee-4e53-b7eb-b9b36cd20943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.314012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314547-b8ee-4e53-b7eb-b9b36cd20943 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.314292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314547-b8ee-4e53-b7eb-b9b36cd20943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.314633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314547-b8ee-4e53-b7eb-b9b36cd20943 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.315262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7314547-b8ee-4e53-b7eb-b9b36cd20943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.316858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7314547-b8ee-4e53-b7eb-b9b36cd20943 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.326680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 802/1595] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:37.327795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5772-885f-4aef-895a-0574ad741259 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.328074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5772-885f-4aef-895a-0574ad741259 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.328912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5772-885f-4aef-895a-0574ad741259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.355395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 794/1596] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:37 2022] PUT /identity/v3/projects/fb6cb03abf5242b594efcf325909dd32/users/fafa609d94334db6907b401e66cb3448/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:37.356642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa0d6e-5546-4618-ada7-bbadccda4f36 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.356927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa0d6e-5546-4618-ada7-bbadccda4f36 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.357202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa0d6e-5546-4618-ada7-bbadccda4f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.363034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-fb1f5772-885f-4aef-895a-0574ad741259 None None] MFA Rules not processed for user `b038a963f86849a5b755b24f4bf202f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:37.364240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fb1f5772-885f-4aef-895a-0574ad741259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:37.376583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d6aa0d6e-5546-4618-ada7-bbadccda4f36 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:37.377824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d6aa0d6e-5546-4618-ada7-bbadccda4f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:37.414639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:37.432233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 803/1597] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:37.433073 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f5772-885f-4aef-895a-0574ad741259 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.437461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 795/1598] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:37.439408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa0d6e-5546-4618-ada7-bbadccda4f36 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.443992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f5772-885f-4aef-895a-0574ad741259 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.448656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf9465-8165-4736-a573-b5a30139a5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.449260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf9465-8165-4736-a573-b5a30139a5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.449542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf9465-8165-4736-a573-b5a30139a5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.449858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf9465-8165-4736-a573-b5a30139a5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.450416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa0d6e-5546-4618-ada7-bbadccda4f36 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.450546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf9465-8165-4736-a573-b5a30139a5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.452098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf9465-8165-4736-a573-b5a30139a5d4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.456358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81448ab-7287-4046-96c9-6fb0363d5408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.457011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81448ab-7287-4046-96c9-6fb0363d5408 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.457289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81448ab-7287-4046-96c9-6fb0363d5408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.457565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81448ab-7287-4046-96c9-6fb0363d5408 admin admin] PATH_INFO: `/v3/projects/fd843c9df17449e59e3545598686ca05/users/76830822b78f4ce7bad5dc87d1b53229/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.463364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 804/1599] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:37.464050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf9465-8165-4736-a573-b5a30139a5d4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.474201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf9465-8165-4736-a573-b5a30139a5d4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.479100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81448ab-7287-4046-96c9-6fb0363d5408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd843c9df17449e59e3545598686ca05, user_id=76830822b78f4ce7bad5dc87d1b53229, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.479400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca86cbf-5200-4362-9bcc-d9d7c5048554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.479996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca86cbf-5200-4362-9bcc-d9d7c5048554 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.480279 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca86cbf-5200-4362-9bcc-d9d7c5048554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.480556 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca86cbf-5200-4362-9bcc-d9d7c5048554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.481195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca86cbf-5200-4362-9bcc-d9d7c5048554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.481831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81448ab-7287-4046-96c9-6fb0363d5408 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.482864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca86cbf-5200-4362-9bcc-d9d7c5048554 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.492993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 805/1600] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:37.493794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca86cbf-5200-4362-9bcc-d9d7c5048554 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.503867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca86cbf-5200-4362-9bcc-d9d7c5048554 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.508418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611cb58-28c6-48bc-8ad4-d00d5f21b140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.509040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611cb58-28c6-48bc-8ad4-d00d5f21b140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.513482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611cb58-28c6-48bc-8ad4-d00d5f21b140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.513482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611cb58-28c6-48bc-8ad4-d00d5f21b140 admin admin] PATH_INFO: `/v3/projects/5b4fba22f55549188446d54f94aa1620/users/c182a0a100054e7681d27d3439aa03f1/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.524641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611cb58-28c6-48bc-8ad4-d00d5f21b140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4fba22f55549188446d54f94aa1620, user_id=c182a0a100054e7681d27d3439aa03f1, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.527304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611cb58-28c6-48bc-8ad4-d00d5f21b140 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.568052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 796/1601] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:37 2022] PUT /identity/v3/projects/fd843c9df17449e59e3545598686ca05/users/76830822b78f4ce7bad5dc87d1b53229/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:37.568856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81448ab-7287-4046-96c9-6fb0363d5408 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.602269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 806/1602] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:37 2022] PUT /identity/v3/projects/5b4fba22f55549188446d54f94aa1620/users/c182a0a100054e7681d27d3439aa03f1/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:37.603525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd728-f50d-452b-8008-73c6e9017dc5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.603807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd728-f50d-452b-8008-73c6e9017dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.604079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd728-f50d-452b-8008-73c6e9017dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.623101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d46bd728-f50d-452b-8008-73c6e9017dc5 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:37.624350 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d46bd728-f50d-452b-8008-73c6e9017dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:37.634117 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81448ab-7287-4046-96c9-6fb0363d5408 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.637809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:37.690473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 807/1603] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:37.691335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bd728-f50d-452b-8008-73c6e9017dc5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.692717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d46bd728-f50d-452b-8008-73c6e9017dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:37.698660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32128e6-8391-48ef-9880-190d45b93f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.699359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32128e6-8391-48ef-9880-190d45b93f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.699642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32128e6-8391-48ef-9880-190d45b93f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.700001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32128e6-8391-48ef-9880-190d45b93f54 admin admin] PATH_INFO: `/v3/projects/8ee90400f90747cc8fb9d6cd30f33eac/users/406e4f40a66340feb8d0769515bbf20d/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.716852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32128e6-8391-48ef-9880-190d45b93f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee90400f90747cc8fb9d6cd30f33eac, user_id=406e4f40a66340feb8d0769515bbf20d, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.719900 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32128e6-8391-48ef-9880-190d45b93f54 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.749742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bd728-f50d-452b-8008-73c6e9017dc5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.753950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332b6b7-8c96-4183-8768-d5b936398b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.754545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332b6b7-8c96-4183-8768-d5b936398b12 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.754822 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332b6b7-8c96-4183-8768-d5b936398b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.755133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332b6b7-8c96-4183-8768-d5b936398b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.763888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332b6b7-8c96-4183-8768-d5b936398b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.765312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332b6b7-8c96-4183-8768-d5b936398b12 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.774858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 808/1604] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:37.775629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332b6b7-8c96-4183-8768-d5b936398b12 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.785527 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 797/1605] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:37 2022] PUT /identity/v3/projects/8ee90400f90747cc8fb9d6cd30f33eac/users/406e4f40a66340feb8d0769515bbf20d/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:37.786393 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32128e6-8391-48ef-9880-190d45b93f54 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.841840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:37.847053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332b6b7-8c96-4183-8768-d5b936398b12 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.864946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32128e6-8391-48ef-9880-190d45b93f54 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:37.870812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead0f5e-2363-4a78-9e42-79b1f547266f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.871559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead0f5e-2363-4a78-9e42-79b1f547266f admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.871899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead0f5e-2363-4a78-9e42-79b1f547266f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.872227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead0f5e-2363-4a78-9e42-79b1f547266f admin admin] PATH_INFO: `/v3/projects/294abb2085b844a99908242d4b02e263/users/26995363963a4a52acc3f57ce1575e6e/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.888590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead0f5e-2363-4a78-9e42-79b1f547266f admin admin] RBAC: Authorizing `identity:create_grant(project_id=294abb2085b844a99908242d4b02e263, user_id=26995363963a4a52acc3f57ce1575e6e, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.891556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead0f5e-2363-4a78-9e42-79b1f547266f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.908623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96b9e6-21b8-4afe-a712-2864079c0162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:37.909262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96b9e6-21b8-4afe-a712-2864079c0162 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.911631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96b9e6-21b8-4afe-a712-2864079c0162 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.911631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96b9e6-21b8-4afe-a712-2864079c0162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:37.970062 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96b9e6-21b8-4afe-a712-2864079c0162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:37.971501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96b9e6-21b8-4afe-a712-2864079c0162 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:37.980451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 809/1606] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:37.981243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96b9e6-21b8-4afe-a712-2864079c0162 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:37.989780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 798/1607] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:37 2022] PUT /identity/v3/projects/294abb2085b844a99908242d4b02e263/users/26995363963a4a52acc3f57ce1575e6e/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:37.992160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214db6da-2364-44c7-b74f-9563a8fd97cf None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:37.992419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214db6da-2364-44c7-b74f-9563a8fd97cf None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:37.992773 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214db6da-2364-44c7-b74f-9563a8fd97cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.001883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96b9e6-21b8-4afe-a712-2864079c0162 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:38.035446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-214db6da-2364-44c7-b74f-9563a8fd97cf None None] MFA Rules not processed for user `76830822b78f4ce7bad5dc87d1b53229`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:38.036703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-214db6da-2364-44c7-b74f-9563a8fd97cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:38.076040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06da8d-a2b8-489a-953f-614b50b671ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:38.076548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06da8d-a2b8-489a-953f-614b50b671ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.076875 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06da8d-a2b8-489a-953f-614b50b671ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.077157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06da8d-a2b8-489a-953f-614b50b671ad admin admin] PATH_INFO: `/v3/projects/12eeba70d923445b901341ebcf58a05e/users/8af07cadb5c548978c00240a5b3e2a08/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.097226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06da8d-a2b8-489a-953f-614b50b671ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=12eeba70d923445b901341ebcf58a05e, user_id=8af07cadb5c548978c00240a5b3e2a08, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:38.099975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06da8d-a2b8-489a-953f-614b50b671ad admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:38.126385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 799/1608] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:38.127933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600542d-023a-40db-877f-dad79e7c5e7f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.128334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600542d-023a-40db-877f-dad79e7c5e7f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.128705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600542d-023a-40db-877f-dad79e7c5e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.136556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:38.149203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b600542d-023a-40db-877f-dad79e7c5e7f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:38.150585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b600542d-023a-40db-877f-dad79e7c5e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:38.161914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 800/1609] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:38.161914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600542d-023a-40db-877f-dad79e7c5e7f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:38.229499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600542d-023a-40db-877f-dad79e7c5e7f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:38.241702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 810/1610] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:37 2022] PUT /identity/v3/projects/12eeba70d923445b901341ebcf58a05e/users/8af07cadb5c548978c00240a5b3e2a08/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:38.241702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e96b5-35bf-4da7-94c2-82a72df00b2c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.241702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e96b5-35bf-4da7-94c2-82a72df00b2c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.241702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e96b5-35bf-4da7-94c2-82a72df00b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.279008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0d9e96b5-35bf-4da7-94c2-82a72df00b2c None None] MFA Rules not processed for user `406e4f40a66340feb8d0769515bbf20d`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:38.279008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0d9e96b5-35bf-4da7-94c2-82a72df00b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:38.299715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31a216-8439-4a9c-9e2b-58e4848c3b6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:38.300324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31a216-8439-4a9c-9e2b-58e4848c3b6d service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.300597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31a216-8439-4a9c-9e2b-58e4848c3b6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.300863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31a216-8439-4a9c-9e2b-58e4848c3b6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.314607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:38.358158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 811/1611] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:38.361270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a465620-4703-4c9b-a232-fb7a21166d7e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.361270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a465620-4703-4c9b-a232-fb7a21166d7e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.361270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a465620-4703-4c9b-a232-fb7a21166d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.382959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31a216-8439-4a9c-9e2b-58e4848c3b6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:38.385021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31a216-8439-4a9c-9e2b-58e4848c3b6d service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:38.394346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3a465620-4703-4c9b-a232-fb7a21166d7e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:38.395582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3a465620-4703-4c9b-a232-fb7a21166d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:38.396524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 801/1612] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:38.397531 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31a216-8439-4a9c-9e2b-58e4848c3b6d service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:38.409146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31a216-8439-4a9c-9e2b-58e4848c3b6d service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:38.413180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 812/1613] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:38.414259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a465620-4703-4c9b-a232-fb7a21166d7e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:38.418461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05198a33-5f7f-4814-b319-20578702c8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:38.419131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05198a33-5f7f-4814-b319-20578702c8ad admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.419414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05198a33-5f7f-4814-b319-20578702c8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.419738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05198a33-5f7f-4814-b319-20578702c8ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.420372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05198a33-5f7f-4814-b319-20578702c8ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:38.424194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05198a33-5f7f-4814-b319-20578702c8ad admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:38.428102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a465620-4703-4c9b-a232-fb7a21166d7e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:38.435249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a8370-dc76-4cd9-85c9-10071cc4440e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:38.436054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a8370-dc76-4cd9-85c9-10071cc4440e service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.436341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a8370-dc76-4cd9-85c9-10071cc4440e service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.438471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a8370-dc76-4cd9-85c9-10071cc4440e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.446919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a8370-dc76-4cd9-85c9-10071cc4440e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:38.448364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a8370-dc76-4cd9-85c9-10071cc4440e service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:38.459158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 813/1614] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:38.471076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c60b1-4289-4262-a911-ff0ef6953599 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.472692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c60b1-4289-4262-a911-ff0ef6953599 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.472692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c60b1-4289-4262-a911-ff0ef6953599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.499525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-925c60b1-4289-4262-a911-ff0ef6953599 None None] MFA Rules not processed for user `8af07cadb5c548978c00240a5b3e2a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:38.500863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-925c60b1-4289-4262-a911-ff0ef6953599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:38.569499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 802/1615] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:38 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:38.570854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf90f8-a2ba-48fe-9adc-3afc8f033ce5 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.571181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf90f8-a2ba-48fe-9adc-3afc8f033ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.571486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf90f8-a2ba-48fe-9adc-3afc8f033ce5 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.572853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 803/1616] 10.222.0.42 () {54 vars in 965 bytes} [Sat Sep 17 03:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:48:38.575191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf90f8-a2ba-48fe-9adc-3afc8f033ce5 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:38.591157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 814/1617] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:38.592184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c60b1-4289-4262-a911-ff0ef6953599 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:38.604388 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c60b1-4289-4262-a911-ff0ef6953599 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:38.632101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:38.646310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf90f8-a2ba-48fe-9adc-3afc8f033ce5 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:38.650676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc003b-c222-49ff-9fbf-f63279e02c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:38.650816 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc003b-c222-49ff-9fbf-f63279e02c8d service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.651080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc003b-c222-49ff-9fbf-f63279e02c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.651354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc003b-c222-49ff-9fbf-f63279e02c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.667658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4573970-ef54-4092-9a3d-6f1045f60cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:38.668288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4573970-ef54-4092-9a3d-6f1045f60cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.668646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4573970-ef54-4092-9a3d-6f1045f60cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.669345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4573970-ef54-4092-9a3d-6f1045f60cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.669597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4573970-ef54-4092-9a3d-6f1045f60cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:38.671948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4573970-ef54-4092-9a3d-6f1045f60cf4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:38.722399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc003b-c222-49ff-9fbf-f63279e02c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:38.723946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc003b-c222-49ff-9fbf-f63279e02c8d service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:38.734019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 804/1618] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:38.735406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55df63-675d-42f0-9f12-5d28f8f36c0e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.735693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55df63-675d-42f0-9f12-5d28f8f36c0e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.735970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55df63-675d-42f0-9f12-5d28f8f36c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.779100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-bb55df63-675d-42f0-9f12-5d28f8f36c0e None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:38.780334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bb55df63-675d-42f0-9f12-5d28f8f36c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:38.812925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 815/1619] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:38 2022] POST /identity/v3/users => generated 295 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:38.812925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4573970-ef54-4092-9a3d-6f1045f60cf4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:38.826618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4573970-ef54-4092-9a3d-6f1045f60cf4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:38.833241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffabe54-c17b-49f2-a523-c4ed7868c9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:38.833898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffabe54-c17b-49f2-a523-c4ed7868c9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.834186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffabe54-c17b-49f2-a523-c4ed7868c9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.834472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffabe54-c17b-49f2-a523-c4ed7868c9f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.835258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffabe54-c17b-49f2-a523-c4ed7868c9f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:38.837511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffabe54-c17b-49f2-a523-c4ed7868c9f1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:38.867146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 805/1620] 10.222.0.42 () {58 vars in 1062 bytes} [Sat Sep 17 03:48:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:38.868133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55df63-675d-42f0-9f12-5d28f8f36c0e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:38.879967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55df63-675d-42f0-9f12-5d28f8f36c0e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:38.885135 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6633f-c996-43cf-9c70-2a1e221c5bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:38.885884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6633f-c996-43cf-9c70-2a1e221c5bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.886209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6633f-c996-43cf-9c70-2a1e221c5bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.886601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6633f-c996-43cf-9c70-2a1e221c5bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.887634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6633f-c996-43cf-9c70-2a1e221c5bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:38.889133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6633f-c996-43cf-9c70-2a1e221c5bc1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:38.905761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 806/1621] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:38.906918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6633f-c996-43cf-9c70-2a1e221c5bc1 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:38.922117 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6633f-c996-43cf-9c70-2a1e221c5bc1 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:38.932693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d6c3d-b842-4731-8642-6ae725f6d419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:38.933652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d6c3d-b842-4731-8642-6ae725f6d419 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:38.934482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d6c3d-b842-4731-8642-6ae725f6d419 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:38.934981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d6c3d-b842-4731-8642-6ae725f6d419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:38.983951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 816/1622] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:38 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:38.986045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffabe54-c17b-49f2-a523-c4ed7868c9f1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:39.013718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d6c3d-b842-4731-8642-6ae725f6d419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:39.015374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d6c3d-b842-4731-8642-6ae725f6d419 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:39.025907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 807/1623] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:39.027071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d6c3d-b842-4731-8642-6ae725f6d419 service nova] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:39.039964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d6c3d-b842-4731-8642-6ae725f6d419 service nova] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:39.064388 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffabe54-c17b-49f2-a523-c4ed7868c9f1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:39.075525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e35208-2fee-4e5a-9bf4-10a4d6868256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:39.076439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e35208-2fee-4e5a-9bf4-10a4d6868256 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.077110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e35208-2fee-4e5a-9bf4-10a4d6868256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.077492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e35208-2fee-4e5a-9bf4-10a4d6868256 admin admin] PATH_INFO: `/v3/projects/d7567d601e194c32bb27e4e0c76f8702/users/2ef5b17ce8d34c76bd65bf70665c8451/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.102563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e35208-2fee-4e5a-9bf4-10a4d6868256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7567d601e194c32bb27e4e0c76f8702, user_id=2ef5b17ce8d34c76bd65bf70665c8451, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:39.105757 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e35208-2fee-4e5a-9bf4-10a4d6868256 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:39.156039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490fbd5-3cd6-41a8-b61a-038615e3c129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:39.157868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490fbd5-3cd6-41a8-b61a-038615e3c129 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.158198 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490fbd5-3cd6-41a8-b61a-038615e3c129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.158513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490fbd5-3cd6-41a8-b61a-038615e3c129 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.159157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490fbd5-3cd6-41a8-b61a-038615e3c129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:39.165303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490fbd5-3cd6-41a8-b61a-038615e3c129 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:39.198871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:39.209100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 817/1624] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:38 2022] PUT /identity/v3/projects/d7567d601e194c32bb27e4e0c76f8702/users/2ef5b17ce8d34c76bd65bf70665c8451/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:39.214006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e35208-2fee-4e5a-9bf4-10a4d6868256 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:39.305778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 808/1625] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:39 2022] POST /identity/v3/users => generated 297 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:39.308946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490fbd5-3cd6-41a8-b61a-038615e3c129 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:39.327278 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490fbd5-3cd6-41a8-b61a-038615e3c129 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:39.385316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e35208-2fee-4e5a-9bf4-10a4d6868256 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:39.395654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e126e4-c0f4-4fa6-9989-21049d8333f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:39.396567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e126e4-c0f4-4fa6-9989-21049d8333f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.396858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e126e4-c0f4-4fa6-9989-21049d8333f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.397315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e126e4-c0f4-4fa6-9989-21049d8333f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.401499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e126e4-c0f4-4fa6-9989-21049d8333f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:39.403996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e126e4-c0f4-4fa6-9989-21049d8333f2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:39.414507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf0ff6-2ee6-4f28-a919-d3ab040adc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:39.415452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf0ff6-2ee6-4f28-a919-d3ab040adc8d admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.415739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf0ff6-2ee6-4f28-a919-d3ab040adc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.416057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf0ff6-2ee6-4f28-a919-d3ab040adc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.416713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf0ff6-2ee6-4f28-a919-d3ab040adc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:39.421212 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf0ff6-2ee6-4f28-a919-d3ab040adc8d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:39.430896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 818/1626] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:48:39.439758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e126e4-c0f4-4fa6-9989-21049d8333f2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:39.450242 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 809/1627] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:39.451162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf0ff6-2ee6-4f28-a919-d3ab040adc8d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:39.463636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf0ff6-2ee6-4f28-a919-d3ab040adc8d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:39.463893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e126e4-c0f4-4fa6-9989-21049d8333f2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:39.471383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad93f3c-7e16-4822-9108-72a1a82bdf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:39.472253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad93f3c-7e16-4822-9108-72a1a82bdf26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.472686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad93f3c-7e16-4822-9108-72a1a82bdf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.473110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad93f3c-7e16-4822-9108-72a1a82bdf26 admin admin] PATH_INFO: `/v3/projects/b2b0cb3bae844af681c7f6146733ba6f/users/759f1c920f124a1eb23d2168903b518c/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.476574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9707e-2dce-42fb-ac26-b7e82d4c1d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:39.478361 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9707e-2dce-42fb-ac26-b7e82d4c1d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.478654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9707e-2dce-42fb-ac26-b7e82d4c1d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.478938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9707e-2dce-42fb-ac26-b7e82d4c1d11 admin admin] PATH_INFO: `/v3/projects/d7567d601e194c32bb27e4e0c76f8702/users/2ef5b17ce8d34c76bd65bf70665c8451/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.480948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9707e-2dce-42fb-ac26-b7e82d4c1d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7567d601e194c32bb27e4e0c76f8702, user_id=2ef5b17ce8d34c76bd65bf70665c8451, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:39.492170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9707e-2dce-42fb-ac26-b7e82d4c1d11 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:39.511645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad93f3c-7e16-4822-9108-72a1a82bdf26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b0cb3bae844af681c7f6146733ba6f, user_id=759f1c920f124a1eb23d2168903b518c, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:39.514590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad93f3c-7e16-4822-9108-72a1a82bdf26 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:39.515130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 819/1628] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:39 2022] PUT /identity/v3/projects/d7567d601e194c32bb27e4e0c76f8702/users/2ef5b17ce8d34c76bd65bf70665c8451/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:39.517591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7051e41-4253-4a60-9a8d-ae2ca6c1370d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.517919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7051e41-4253-4a60-9a8d-ae2ca6c1370d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.518270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7051e41-4253-4a60-9a8d-ae2ca6c1370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.559028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a7051e41-4253-4a60-9a8d-ae2ca6c1370d None None] MFA Rules not processed for user `2ef5b17ce8d34c76bd65bf70665c8451`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:39.560383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a7051e41-4253-4a60-9a8d-ae2ca6c1370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:39.645841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 820/1629] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:39.654258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:39.654976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7051e41-4253-4a60-9a8d-ae2ca6c1370d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:39.658027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 810/1630] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:39 2022] PUT /identity/v3/projects/b2b0cb3bae844af681c7f6146733ba6f/users/759f1c920f124a1eb23d2168903b518c/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:39.660241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad93f3c-7e16-4822-9108-72a1a82bdf26 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:39.717940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7051e41-4253-4a60-9a8d-ae2ca6c1370d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:39.751046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad93f3c-7e16-4822-9108-72a1a82bdf26 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:39.759173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273aa4-819f-49f1-85c1-df99d4dfb791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:39.759844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273aa4-819f-49f1-85c1-df99d4dfb791 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.760151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273aa4-819f-49f1-85c1-df99d4dfb791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.760434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273aa4-819f-49f1-85c1-df99d4dfb791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.765333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e273aa4-819f-49f1-85c1-df99d4dfb791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:39.765333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e273aa4-819f-49f1-85c1-df99d4dfb791 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:39.783692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca9cc3-4fb0-485d-8139-675d26911ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:39.783902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 811/1631] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:39.786906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9cc3-4fb0-485d-8139-675d26911ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.786906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9cc3-4fb0-485d-8139-675d26911ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.786906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9cc3-4fb0-485d-8139-675d26911ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.812181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273aa4-819f-49f1-85c1-df99d4dfb791 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:39.838031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273aa4-819f-49f1-85c1-df99d4dfb791 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:39.850797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d463d-1308-4222-8619-9706d109b9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:39.851475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d463d-1308-4222-8619-9706d109b9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.851811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d463d-1308-4222-8619-9706d109b9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.852010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d463d-1308-4222-8619-9706d109b9b5 admin admin] PATH_INFO: `/v3/projects/b2b0cb3bae844af681c7f6146733ba6f/users/759f1c920f124a1eb23d2168903b518c/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.857239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d463d-1308-4222-8619-9706d109b9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b0cb3bae844af681c7f6146733ba6f, user_id=759f1c920f124a1eb23d2168903b518c, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:39.860286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d463d-1308-4222-8619-9706d109b9b5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:39.874303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca9cc3-4fb0-485d-8139-675d26911ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:39.876619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca9cc3-4fb0-485d-8139-675d26911ae6 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:39.884631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 812/1632] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:39 2022] PUT /identity/v3/projects/b2b0cb3bae844af681c7f6146733ba6f/users/759f1c920f124a1eb23d2168903b518c/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:39.888050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3bb2-140e-4d70-9184-111c9f6f5a72 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.888599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3bb2-140e-4d70-9184-111c9f6f5a72 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.890204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3bb2-140e-4d70-9184-111c9f6f5a72 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.893040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 813/1633] 10.222.0.42 () {54 vars in 965 bytes} [Sat Sep 17 03:48:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:48:39.897486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 821/1634] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:39.901463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f168ba-8b69-4d61-947c-f93f539d7daa None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.902064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f168ba-8b69-4d61-947c-f93f539d7daa None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.909204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f168ba-8b69-4d61-947c-f93f539d7daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.909675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717ca09-ad87-47a1-8761-8f9ffb1e057d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:39.909675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717ca09-ad87-47a1-8761-8f9ffb1e057d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:39.909675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717ca09-ad87-47a1-8761-8f9ffb1e057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:39.944039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-38f168ba-8b69-4d61-947c-f93f539d7daa None None] MFA Rules not processed for user `759f1c920f124a1eb23d2168903b518c`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:39.945580 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-38f168ba-8b69-4d61-947c-f93f539d7daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:39.956156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c717ca09-ad87-47a1-8761-8f9ffb1e057d None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:39.959047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c717ca09-ad87-47a1-8761-8f9ffb1e057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:40.087332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 822/1635] 10.222.0.42 () {58 vars in 1062 bytes} [Sat Sep 17 03:48:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:40.095263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dd3fe-ef8b-47f7-996a-9ae933b25fe2 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:40.095912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dd3fe-ef8b-47f7-996a-9ae933b25fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:40.096378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dd3fe-ef8b-47f7-996a-9ae933b25fe2 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:40.099743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 823/1636] 10.222.0.42 () {54 vars in 912 bytes} [Sat Sep 17 03:48:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:48:40.105463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d40ec-849b-44aa-aadb-46cd682a3f8e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:40.106046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d40ec-849b-44aa-aadb-46cd682a3f8e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:40.106544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d40ec-849b-44aa-aadb-46cd682a3f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:40.119446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 814/1637] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:48:40.125314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:40.396186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f168ba-8b69-4d61-947c-f93f539d7daa None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:40.397535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-074d40ec-849b-44aa-aadb-46cd682a3f8e None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:40.397535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-074d40ec-849b-44aa-aadb-46cd682a3f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:40.397535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 824/1638] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:48:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:40.399126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f168ba-8b69-4d61-947c-f93f539d7daa None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:40.399126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b542d4b-40a9-4a42-95a0-2e01a9c8dbc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:40.399126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b542d4b-40a9-4a42-95a0-2e01a9c8dbc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:40.399126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b542d4b-40a9-4a42-95a0-2e01a9c8dbc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:40.399977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b542d4b-40a9-4a42-95a0-2e01a9c8dbc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:40.399977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b542d4b-40a9-4a42-95a0-2e01a9c8dbc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:40.399977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b542d4b-40a9-4a42-95a0-2e01a9c8dbc8 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:40.399977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 815/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:48:40.399977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:40.521833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d40ec-849b-44aa-aadb-46cd682a3f8e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:40.543432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d40ec-849b-44aa-aadb-46cd682a3f8e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:40.555882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead338dc-b9ad-46cf-9e30-666488bb1ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:40.557274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead338dc-b9ad-46cf-9e30-666488bb1ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:40.559074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead338dc-b9ad-46cf-9e30-666488bb1ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:40.559555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead338dc-b9ad-46cf-9e30-666488bb1ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:40.602173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead338dc-b9ad-46cf-9e30-666488bb1ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:40.609576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead338dc-b9ad-46cf-9e30-666488bb1ad8 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:40.634032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 825/1640] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:40.741318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dd3f3-49ab-4f55-8c71-31eda4ba9ab1 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:40.741606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dd3f3-49ab-4f55-8c71-31eda4ba9ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:40.741916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dd3f3-49ab-4f55-8c71-31eda4ba9ab1 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:40.743730 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 816/1641] 10.222.0.42 () {54 vars in 912 bytes} [Sat Sep 17 03:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:48:40.763426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daecda-54fb-4f59-834b-5b720aaf18cd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:40.763934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daecda-54fb-4f59-834b-5b720aaf18cd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:40.764811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daecda-54fb-4f59-834b-5b720aaf18cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:40.789009 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f4daecda-54fb-4f59-834b-5b720aaf18cd None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:40.790566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f4daecda-54fb-4f59-834b-5b720aaf18cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:40.813949 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 826/1642] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:48:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:40.935990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dd3f3-49ab-4f55-8c71-31eda4ba9ab1 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:40.962847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dd3f3-49ab-4f55-8c71-31eda4ba9ab1 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:40.970281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f28944-1d66-423b-9ae9-2fd862a8a748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:40.970906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f28944-1d66-423b-9ae9-2fd862a8a748 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:40.971237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f28944-1d66-423b-9ae9-2fd862a8a748 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:40.971524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f28944-1d66-423b-9ae9-2fd862a8a748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:40.990518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daecda-54fb-4f59-834b-5b720aaf18cd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:40.991295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f28944-1d66-423b-9ae9-2fd862a8a748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:40.993980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f28944-1d66-423b-9ae9-2fd862a8a748 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:41.008127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daecda-54fb-4f59-834b-5b720aaf18cd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:41.014018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ba6b2-0442-43bc-b2ab-40489d8b7f8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:41.014757 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ba6b2-0442-43bc-b2ab-40489d8b7f8d service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:41.015142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ba6b2-0442-43bc-b2ab-40489d8b7f8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:41.015533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ba6b2-0442-43bc-b2ab-40489d8b7f8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:41.026159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 817/1643] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:41.150130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ba6b2-0442-43bc-b2ab-40489d8b7f8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:41.156365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ba6b2-0442-43bc-b2ab-40489d8b7f8d service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:41.180341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 827/1644] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:41.902656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f28944-1d66-423b-9ae9-2fd862a8a748 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:41.979684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f28944-1d66-423b-9ae9-2fd862a8a748 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:41.985958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:41.986465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:41.986748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:41.987026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] PATH_INFO: `/v3/users/406e4f40a66340feb8d0769515bbf20d` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:41.988351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=406e4f40a66340feb8d0769515bbf20d)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:41.990642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:42.248799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '406e4f40a66340feb8d0769515bbf20d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:48:42.304578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:42.374483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '406e4f40a66340feb8d0769515bbf20d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:48:42.376151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '406e4f40a66340feb8d0769515bbf20d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:48:42.385427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '406e4f40a66340feb8d0769515bbf20d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:48:42.409871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '406e4f40a66340feb8d0769515bbf20d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:48:42.421606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 818/1645] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:48:41 2022] DELETE /identity/v3/users/406e4f40a66340feb8d0769515bbf20d => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:42.423966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d11b6-6ce0-4383-81f1-52bd80ebdb5c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:42.424257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d11b6-6ce0-4383-81f1-52bd80ebdb5c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:42.424598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d11b6-6ce0-4383-81f1-52bd80ebdb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:42.448688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8f7d11b6-6ce0-4383-81f1-52bd80ebdb5c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:42.452683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8f7d11b6-6ce0-4383-81f1-52bd80ebdb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:42.546114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:42.568079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 828/1646] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:42.588376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:42.590060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:42.819403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031eb6b-bbe3-4966-b0aa-58ba87710a76 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:42.835869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd81dc-cfd2-4bd2-92bd-012d0c9e3eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:42.837600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd81dc-cfd2-4bd2-92bd-012d0c9e3eef service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:42.837947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd81dc-cfd2-4bd2-92bd-012d0c9e3eef service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:42.838279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd81dc-cfd2-4bd2-92bd-012d0c9e3eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:42.857506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd81dc-cfd2-4bd2-92bd-012d0c9e3eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:42.860849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd81dc-cfd2-4bd2-92bd-012d0c9e3eef service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:42.897466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 819/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:44.744675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d11b6-6ce0-4383-81f1-52bd80ebdb5c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:44.747148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8f7d11b6-6ce0-4383-81f1-52bd80ebdb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:44.789848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d11b6-6ce0-4383-81f1-52bd80ebdb5c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:44.818286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:44.822878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:44.823287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:44.823580 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] PATH_INFO: `/v3/projects/8ee90400f90747cc8fb9d6cd30f33eac` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:44.828520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee90400f90747cc8fb9d6cd30f33eac)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:44.831217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:45.085453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee90400f90747cc8fb9d6cd30f33eac', 'request_id': 'req-b40a51be-b6fa-4633-812d-922dc8004994'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:48:45.106934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee90400f90747cc8fb9d6cd30f33eac', 'request_id': 'req-b40a51be-b6fa-4633-812d-922dc8004994'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:48:45.235787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee90400f90747cc8fb9d6cd30f33eac', 'request_id': 'req-b40a51be-b6fa-4633-812d-922dc8004994'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:48:45.251239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:45.258497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] The token cache is being invalidate because project 8ee90400f90747cc8fb9d6cd30f33eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:48:45.264341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:48:45.266006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b40a51be-b6fa-4633-812d-922dc8004994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:48:45.268577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 829/1648] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:48:44 2022] DELETE /identity/v3/projects/8ee90400f90747cc8fb9d6cd30f33eac => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:45.276295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3891976-003c-4e30-bec1-420a6053ae85 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:45.276778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3891976-003c-4e30-bec1-420a6053ae85 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:45.277067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3891976-003c-4e30-bec1-420a6053ae85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:45.321315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b3891976-003c-4e30-bec1-420a6053ae85 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:45.322830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b3891976-003c-4e30-bec1-420a6053ae85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:45.526095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 820/1649] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:45.537455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f0c02-ee36-4fe6-bc80-953e889b67ad None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:45.537917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f0c02-ee36-4fe6-bc80-953e889b67ad None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:45.538793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f0c02-ee36-4fe6-bc80-953e889b67ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:45.579315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-754f0c02-ee36-4fe6-bc80-953e889b67ad None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:45.580726 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-754f0c02-ee36-4fe6-bc80-953e889b67ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:45.603525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3891976-003c-4e30-bec1-420a6053ae85 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:45.612957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 830/1650] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:45.619447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3891976-003c-4e30-bec1-420a6053ae85 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:45.624694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c449-09aa-43d3-b5c9-2fffcd860f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:45.626538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c449-09aa-43d3-b5c9-2fffcd860f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:45.627023 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c449-09aa-43d3-b5c9-2fffcd860f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:45.627489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c449-09aa-43d3-b5c9-2fffcd860f93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:45.628408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5c449-09aa-43d3-b5c9-2fffcd860f93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:48:45.628936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5c449-09aa-43d3-b5c9-2fffcd860f93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:45.630867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5c449-09aa-43d3-b5c9-2fffcd860f93 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:45.653667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f0c02-ee36-4fe6-bc80-953e889b67ad None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:45.660860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 821/1651] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:48:45.669587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f0c02-ee36-4fe6-bc80-953e889b67ad None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:45.673220 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0521a-9114-4e2a-bd34-5884747ebaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:45.674707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0521a-9114-4e2a-bd34-5884747ebaee admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:45.675239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0521a-9114-4e2a-bd34-5884747ebaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:45.675712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0521a-9114-4e2a-bd34-5884747ebaee admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:45.676613 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0521a-9114-4e2a-bd34-5884747ebaee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:45.679122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0521a-9114-4e2a-bd34-5884747ebaee admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:45.816260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c449-09aa-43d3-b5c9-2fffcd860f93 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:45.823720 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 831/1652] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:45 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:45.824239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:45.832340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c449-09aa-43d3-b5c9-2fffcd860f93 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:46.047642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe0ece-f7cb-45dd-931a-3e67c64707a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:46.050187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe0ece-f7cb-45dd-931a-3e67c64707a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:46.050187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe0ece-f7cb-45dd-931a-3e67c64707a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:46.050187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe0ece-f7cb-45dd-931a-3e67c64707a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:46.050187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe0ece-f7cb-45dd-931a-3e67c64707a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:46.054548 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe0ece-f7cb-45dd-931a-3e67c64707a7 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:46.168123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0521a-9114-4e2a-bd34-5884747ebaee admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:46.169172 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 822/1653] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:45 2022] POST /identity/v3/users => generated 293 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:46.180653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0521a-9114-4e2a-bd34-5884747ebaee admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:46.190220 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32d6bb-00f7-4b5f-b31e-691132954583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:46.191133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32d6bb-00f7-4b5f-b31e-691132954583 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:46.191561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32d6bb-00f7-4b5f-b31e-691132954583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:46.191990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32d6bb-00f7-4b5f-b31e-691132954583 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:46.192795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32d6bb-00f7-4b5f-b31e-691132954583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:46.196599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32d6bb-00f7-4b5f-b31e-691132954583 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:46.220116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe0ece-f7cb-45dd-931a-3e67c64707a7 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:46.221511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 832/1654] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:48:46.246741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe0ece-f7cb-45dd-931a-3e67c64707a7 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:46.253813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5dcd5-d45a-4e8b-a2cb-a4991716e899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:46.254703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5dcd5-d45a-4e8b-a2cb-a4991716e899 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:46.255157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5dcd5-d45a-4e8b-a2cb-a4991716e899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:46.255600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5dcd5-d45a-4e8b-a2cb-a4991716e899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:46.256444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5dcd5-d45a-4e8b-a2cb-a4991716e899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:46.258299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5dcd5-d45a-4e8b-a2cb-a4991716e899 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:46.277114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32d6bb-00f7-4b5f-b31e-691132954583 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:46.284848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 823/1655] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:48:46.292037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32d6bb-00f7-4b5f-b31e-691132954583 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:46.298432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd1e3e-dce7-4cef-8535-ca2e4e16bfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:46.300290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd1e3e-dce7-4cef-8535-ca2e4e16bfa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:46.300766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd1e3e-dce7-4cef-8535-ca2e4e16bfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:46.301224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd1e3e-dce7-4cef-8535-ca2e4e16bfa1 admin admin] PATH_INFO: `/v3/projects/a9f513d1f74f452eba7d4f000bc6f5dd/users/55f2fbf563c4435b8dedc99b3102cde3/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:46.320919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd1e3e-dce7-4cef-8535-ca2e4e16bfa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f513d1f74f452eba7d4f000bc6f5dd, user_id=55f2fbf563c4435b8dedc99b3102cde3, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:46.325023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd1e3e-dce7-4cef-8535-ca2e4e16bfa1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:46.439735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 833/1656] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:46 2022] PUT /identity/v3/projects/a9f513d1f74f452eba7d4f000bc6f5dd/users/55f2fbf563c4435b8dedc99b3102cde3/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:46.442123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fae9d-0614-4644-98a3-fb2601c9993e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:46.443766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fae9d-0614-4644-98a3-fb2601c9993e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:46.444174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fae9d-0614-4644-98a3-fb2601c9993e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:46.504056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-401fae9d-0614-4644-98a3-fb2601c9993e None None] MFA Rules not processed for user `55f2fbf563c4435b8dedc99b3102cde3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:46.505506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-401fae9d-0614-4644-98a3-fb2601c9993e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:46.615536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 824/1657] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:46.617556 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd1e3e-dce7-4cef-8535-ca2e4e16bfa1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:46.705541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd1e3e-dce7-4cef-8535-ca2e4e16bfa1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:46.714266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b31d10-dcfb-4c2b-abf2-cf7bc13c50b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:46.715369 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b31d10-dcfb-4c2b-abf2-cf7bc13c50b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:46.715973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b31d10-dcfb-4c2b-abf2-cf7bc13c50b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:46.716459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b31d10-dcfb-4c2b-abf2-cf7bc13c50b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:46.717426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b31d10-dcfb-4c2b-abf2-cf7bc13c50b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:46.722097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b31d10-dcfb-4c2b-abf2-cf7bc13c50b8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:46.864479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 834/1658] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:46 2022] POST /identity/v3/projects => generated 383 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:46.865986 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fae9d-0614-4644-98a3-fb2601c9993e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:46.889904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fae9d-0614-4644-98a3-fb2601c9993e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:47.024427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b459c5-c658-4bf0-995c-ca2cdeafb091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:47.025130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b459c5-c658-4bf0-995c-ca2cdeafb091 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:47.026872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b459c5-c658-4bf0-995c-ca2cdeafb091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:47.027142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b459c5-c658-4bf0-995c-ca2cdeafb091 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:47.027802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b459c5-c658-4bf0-995c-ca2cdeafb091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:47.033406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b459c5-c658-4bf0-995c-ca2cdeafb091 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:47.188535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 825/1659] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:46 2022] POST /identity/v3/users => generated 292 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:47.194863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b31d10-dcfb-4c2b-abf2-cf7bc13c50b8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:47.211452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b31d10-dcfb-4c2b-abf2-cf7bc13c50b8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:47.227467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951d81f-ddb9-4c88-9bb6-e1f4daee72f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:47.231213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951d81f-ddb9-4c88-9bb6-e1f4daee72f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:47.231715 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951d81f-ddb9-4c88-9bb6-e1f4daee72f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:47.232206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951d81f-ddb9-4c88-9bb6-e1f4daee72f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:47.233631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951d81f-ddb9-4c88-9bb6-e1f4daee72f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:47.240832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951d81f-ddb9-4c88-9bb6-e1f4daee72f9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:47.270686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 835/1660] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:47.274073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b459c5-c658-4bf0-995c-ca2cdeafb091 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:47.289168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b459c5-c658-4bf0-995c-ca2cdeafb091 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:47.303476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72201c-07b5-4d76-b325-a262c240ea0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:47.304354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72201c-07b5-4d76-b325-a262c240ea0e admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:47.304786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72201c-07b5-4d76-b325-a262c240ea0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:47.305236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72201c-07b5-4d76-b325-a262c240ea0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:47.312135 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72201c-07b5-4d76-b325-a262c240ea0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:47.312135 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72201c-07b5-4d76-b325-a262c240ea0e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:47.339901 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951d81f-ddb9-4c88-9bb6-e1f4daee72f9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:47.350705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 826/1661] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:47.354856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951d81f-ddb9-4c88-9bb6-e1f4daee72f9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:47.360891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3d4a6-b0c8-4a68-beda-8aaddc7229b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:47.361864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3d4a6-b0c8-4a68-beda-8aaddc7229b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:47.362350 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3d4a6-b0c8-4a68-beda-8aaddc7229b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:47.362806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3d4a6-b0c8-4a68-beda-8aaddc7229b7 admin admin] PATH_INFO: `/v3/projects/f1a83565f60e4cc4909033e2ded526c4/users/60ef0cd7192e426685c82754f4deba8c/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:47.393634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3d4a6-b0c8-4a68-beda-8aaddc7229b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a83565f60e4cc4909033e2ded526c4, user_id=60ef0cd7192e426685c82754f4deba8c, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:47.398651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3d4a6-b0c8-4a68-beda-8aaddc7229b7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:47.500723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 836/1662] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:47 2022] PUT /identity/v3/projects/f1a83565f60e4cc4909033e2ded526c4/users/60ef0cd7192e426685c82754f4deba8c/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:47.505511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c42f14-16f8-434a-862e-88b1ec9e0243 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:47.506126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c42f14-16f8-434a-862e-88b1ec9e0243 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:47.507055 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c42f14-16f8-434a-862e-88b1ec9e0243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:47.542204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:47.600794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d3c42f14-16f8-434a-862e-88b1ec9e0243 None None] MFA Rules not processed for user `60ef0cd7192e426685c82754f4deba8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:47.602948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d3c42f14-16f8-434a-862e-88b1ec9e0243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:47.714443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 827/1663] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:47.719430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3d4a6-b0c8-4a68-beda-8aaddc7229b7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:47.837405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:47.868847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3d4a6-b0c8-4a68-beda-8aaddc7229b7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:47.878282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82b57d-8d67-4b12-9b1a-a739167222f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:47.879161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82b57d-8d67-4b12-9b1a-a739167222f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:47.885769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82b57d-8d67-4b12-9b1a-a739167222f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:47.885769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82b57d-8d67-4b12-9b1a-a739167222f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:47.885769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82b57d-8d67-4b12-9b1a-a739167222f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:47.885769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82b57d-8d67-4b12-9b1a-a739167222f0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:47.908747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c42f14-16f8-434a-862e-88b1ec9e0243 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:47.918971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d3c42f14-16f8-434a-862e-88b1ec9e0243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:48.007907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c42f14-16f8-434a-862e-88b1ec9e0243 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:48.036271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 837/1664] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:48:47 2022] POST /identity/v3/projects => generated 383 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:48.037480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82b57d-8d67-4b12-9b1a-a739167222f0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:48.038588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aadf4-39cc-4149-90fa-54fef3b60519 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:48.039232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aadf4-39cc-4149-90fa-54fef3b60519 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:48.039515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aadf4-39cc-4149-90fa-54fef3b60519 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:48.039791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aadf4-39cc-4149-90fa-54fef3b60519 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:48.050509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9a4aadf4-39cc-4149-90fa-54fef3b60519 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:48.246903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aadf4-39cc-4149-90fa-54fef3b60519 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:48.252019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aadf4-39cc-4149-90fa-54fef3b60519 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:48.282189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 828/1665] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:48.319427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82b57d-8d67-4b12-9b1a-a739167222f0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:48.326271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e00d3-b2ed-4ebf-a67a-428b84b59e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:48.327925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e00d3-b2ed-4ebf-a67a-428b84b59e5a admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:48.328362 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e00d3-b2ed-4ebf-a67a-428b84b59e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:48.328939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e00d3-b2ed-4ebf-a67a-428b84b59e5a admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:48.329861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e00d3-b2ed-4ebf-a67a-428b84b59e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:48.334240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e00d3-b2ed-4ebf-a67a-428b84b59e5a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:48.452755 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 838/1666] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:48:48 2022] POST /identity/v3/users => generated 298 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:48:48.453989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aadf4-39cc-4149-90fa-54fef3b60519 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:48.475216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aadf4-39cc-4149-90fa-54fef3b60519 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:48.505938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7ddf-95a7-4d18-a8db-46ee994bad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:48.505938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7ddf-95a7-4d18-a8db-46ee994bad4c admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:48.505938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7ddf-95a7-4d18-a8db-46ee994bad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:48.505938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7ddf-95a7-4d18-a8db-46ee994bad4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:48.506215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff7ddf-95a7-4d18-a8db-46ee994bad4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:48.506215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff7ddf-95a7-4d18-a8db-46ee994bad4c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:48.529045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 829/1667] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:48.532974 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e00d3-b2ed-4ebf-a67a-428b84b59e5a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:48.567567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e00d3-b2ed-4ebf-a67a-428b84b59e5a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:48.582175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e34a2-fb04-44e8-b68a-fb882b98b685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:48.583028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e34a2-fb04-44e8-b68a-fb882b98b685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:48.584825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e34a2-fb04-44e8-b68a-fb882b98b685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:48.587821 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e34a2-fb04-44e8-b68a-fb882b98b685 admin admin] PATH_INFO: `/v3/projects/49f2070fe01b4cd89fd59d3463e9482f/users/d0d3b3d878a84a5d8f6e165a7c556499/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:48.680373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e34a2-fb04-44e8-b68a-fb882b98b685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f2070fe01b4cd89fd59d3463e9482f, user_id=d0d3b3d878a84a5d8f6e165a7c556499, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:48.699002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e34a2-fb04-44e8-b68a-fb882b98b685 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:48.813688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 839/1668] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:48 2022] PUT /identity/v3/projects/49f2070fe01b4cd89fd59d3463e9482f/users/d0d3b3d878a84a5d8f6e165a7c556499/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:48.815340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7ddf-95a7-4d18-a8db-46ee994bad4c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:48.840533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7ddf-95a7-4d18-a8db-46ee994bad4c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:48.906607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:49.044207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730f758-0cca-4c17-b780-8a59dd6bd712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:49.044840 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730f758-0cca-4c17-b780-8a59dd6bd712 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:49.045125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730f758-0cca-4c17-b780-8a59dd6bd712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:49.045395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730f758-0cca-4c17-b780-8a59dd6bd712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:49.051445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730f758-0cca-4c17-b780-8a59dd6bd712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:49.052120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730f758-0cca-4c17-b780-8a59dd6bd712 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:49.081238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 830/1669] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:48:49.082036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e34a2-fb04-44e8-b68a-fb882b98b685 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:49.105891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e34a2-fb04-44e8-b68a-fb882b98b685 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:49.117181 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e76647-aad4-4c50-a175-244afa6de99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:49.117900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76647-aad4-4c50-a175-244afa6de99a admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:49.118184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76647-aad4-4c50-a175-244afa6de99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:49.118634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76647-aad4-4c50-a175-244afa6de99a admin admin] PATH_INFO: `/v3/projects/49f2070fe01b4cd89fd59d3463e9482f/users/d0d3b3d878a84a5d8f6e165a7c556499/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:49.125305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e76647-aad4-4c50-a175-244afa6de99a admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f2070fe01b4cd89fd59d3463e9482f, user_id=d0d3b3d878a84a5d8f6e165a7c556499, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:49.134076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e76647-aad4-4c50-a175-244afa6de99a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:49.171726 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 840/1670] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:48:49 2022] PUT /identity/v3/projects/49f2070fe01b4cd89fd59d3463e9482f/users/d0d3b3d878a84a5d8f6e165a7c556499/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:48:49.183080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5a5c-8283-46d7-8fec-523ba3851c8e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:49.183463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5a5c-8283-46d7-8fec-523ba3851c8e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:49.183813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5a5c-8283-46d7-8fec-523ba3851c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:49.263716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-310d5a5c-8283-46d7-8fec-523ba3851c8e None None] MFA Rules not processed for user `d0d3b3d878a84a5d8f6e165a7c556499`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:49.266461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-310d5a5c-8283-46d7-8fec-523ba3851c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:49.498337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 831/1671] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:48:49.583901 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e76647-aad4-4c50-a175-244afa6de99a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:49.585795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-16e76647-aad4-4c50-a175-244afa6de99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:49.728940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e76647-aad4-4c50-a175-244afa6de99a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:49.733553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370163c4-0278-404d-9ad3-b1e015603b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:49.735337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370163c4-0278-404d-9ad3-b1e015603b1c service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:49.735644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370163c4-0278-404d-9ad3-b1e015603b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:49.735971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370163c4-0278-404d-9ad3-b1e015603b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:49.850774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:49.904511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370163c4-0278-404d-9ad3-b1e015603b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:49.907341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370163c4-0278-404d-9ad3-b1e015603b1c service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:49.933868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 841/1672] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:51.212070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8a431-e866-40f1-93f0-8ec11627784b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:51.212543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8a431-e866-40f1-93f0-8ec11627784b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:51.212639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8a431-e866-40f1-93f0-8ec11627784b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:51.296018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-dbd8a431-e866-40f1-93f0-8ec11627784b None None] MFA Rules not processed for user `2723e5c74bde49f485099b615b36997a`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:51.298075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-dbd8a431-e866-40f1-93f0-8ec11627784b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:51.512958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 832/1673] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:51.530404 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370163c4-0278-404d-9ad3-b1e015603b1c service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:51.536074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-370163c4-0278-404d-9ad3-b1e015603b1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:51.758606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370163c4-0278-404d-9ad3-b1e015603b1c service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:51.774461 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae230d-a850-48ed-9192-9b662cf4d773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:51.775176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae230d-a850-48ed-9192-9b662cf4d773 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:51.775620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae230d-a850-48ed-9192-9b662cf4d773 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:51.775814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae230d-a850-48ed-9192-9b662cf4d773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:51.789924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae230d-a850-48ed-9192-9b662cf4d773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:51.790061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae230d-a850-48ed-9192-9b662cf4d773 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:51.826942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 842/1674] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:52.169881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9250a-1d26-4103-ba39-c4509d2469fb None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:52.169881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9250a-1d26-4103-ba39-c4509d2469fb None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:52.169881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9250a-1d26-4103-ba39-c4509d2469fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:52.220143 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-59b9250a-1d26-4103-ba39-c4509d2469fb None None] MFA Rules not processed for user `fafa609d94334db6907b401e66cb3448`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:52.221356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-59b9250a-1d26-4103-ba39-c4509d2469fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:52.268230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:52.372150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 833/1675] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:52.397963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae230d-a850-48ed-9192-9b662cf4d773 service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:52.399555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cbae230d-a850-48ed-9192-9b662cf4d773 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:52.437860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae230d-a850-48ed-9192-9b662cf4d773 service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:52.460151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b83f9-6077-4aba-b0ea-b835d942f953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:52.460878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b83f9-6077-4aba-b0ea-b835d942f953 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:52.461252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b83f9-6077-4aba-b0ea-b835d942f953 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:52.463169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b83f9-6077-4aba-b0ea-b835d942f953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:52.490229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b83f9-6077-4aba-b0ea-b835d942f953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:52.492244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b83f9-6077-4aba-b0ea-b835d942f953 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:52.519848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 843/1676] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:53.344818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2d7b6-9460-412e-ab4a-24f6d4d6d507 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:53.345287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2d7b6-9460-412e-ab4a-24f6d4d6d507 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:53.345396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2d7b6-9460-412e-ab4a-24f6d4d6d507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:53.400850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-36b2d7b6-9460-412e-ab4a-24f6d4d6d507 None None] MFA Rules not processed for user `26995363963a4a52acc3f57ce1575e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:53.402435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-36b2d7b6-9460-412e-ab4a-24f6d4d6d507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:53.600362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 834/1677] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:53.613047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fd9da-8f0a-4206-890f-8ed435b0965c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:53.613585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fd9da-8f0a-4206-890f-8ed435b0965c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:53.613661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fd9da-8f0a-4206-890f-8ed435b0965c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:53.639172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-409fd9da-8f0a-4206-890f-8ed435b0965c None None] MFA Rules not processed for user `26995363963a4a52acc3f57ce1575e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:53.643312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-409fd9da-8f0a-4206-890f-8ed435b0965c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:53.675746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 844/1678] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:53.681170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2d7b6-9460-412e-ab4a-24f6d4d6d507 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:53.697510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2d7b6-9460-412e-ab4a-24f6d4d6d507 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:53.704543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c0cd1-61ce-43b7-b359-98756680ced8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:53.705748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c0cd1-61ce-43b7-b359-98756680ced8 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:53.706173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c0cd1-61ce-43b7-b359-98756680ced8 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:53.706685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c0cd1-61ce-43b7-b359-98756680ced8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:53.717000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c0cd1-61ce-43b7-b359-98756680ced8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:53.720270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c0cd1-61ce-43b7-b359-98756680ced8 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:53.741837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 835/1679] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:54.069443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03134a7-8299-4806-abda-f371fcbe5f76 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:54.069863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03134a7-8299-4806-abda-f371fcbe5f76 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:54.070267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03134a7-8299-4806-abda-f371fcbe5f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:54.131938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c03134a7-8299-4806-abda-f371fcbe5f76 None None] MFA Rules not processed for user `c182a0a100054e7681d27d3439aa03f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:54.133406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c03134a7-8299-4806-abda-f371fcbe5f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:54.321690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 845/1680] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:54.333070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f140e0-22e0-45c6-a721-6a55c944c279 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:54.333355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f140e0-22e0-45c6-a721-6a55c944c279 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:54.333865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f140e0-22e0-45c6-a721-6a55c944c279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:54.382520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-c8f140e0-22e0-45c6-a721-6a55c944c279 None None] MFA Rules not processed for user `c182a0a100054e7681d27d3439aa03f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:54.386058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c8f140e0-22e0-45c6-a721-6a55c944c279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:54.400132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 836/1681] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:54.413360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03134a7-8299-4806-abda-f371fcbe5f76 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:54.446005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03134a7-8299-4806-abda-f371fcbe5f76 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:54.449260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce930cd-ae73-4fba-a1a4-072cf863ba32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:54.449933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce930cd-ae73-4fba-a1a4-072cf863ba32 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:54.450218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce930cd-ae73-4fba-a1a4-072cf863ba32 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:54.450531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce930cd-ae73-4fba-a1a4-072cf863ba32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:54.472947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce930cd-ae73-4fba-a1a4-072cf863ba32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:54.491673 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce930cd-ae73-4fba-a1a4-072cf863ba32 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:54.510172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 846/1682] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:54.658679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db3c2a-4729-494c-a5aa-a75885bab843 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:54.658966 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db3c2a-4729-494c-a5aa-a75885bab843 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:54.659379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db3c2a-4729-494c-a5aa-a75885bab843 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:54.665304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 837/1683] 10.222.0.42 () {54 vars in 912 bytes} [Sat Sep 17 03:48:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:48:54.675379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471a98a-ebe4-4d52-9baf-b2c6ffc42516 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:54.675928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471a98a-ebe4-4d52-9baf-b2c6ffc42516 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:54.675928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471a98a-ebe4-4d52-9baf-b2c6ffc42516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:54.734348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:54.761201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6471a98a-ebe4-4d52-9baf-b2c6ffc42516 None None] MFA Rules not processed for user `6336f0c469b84db78a90cf8435c7dbe4`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:54.761201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6471a98a-ebe4-4d52-9baf-b2c6ffc42516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:54.875839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 847/1684] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:48:54.912952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db3c2a-4729-494c-a5aa-a75885bab843 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:54.917790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b0db3c2a-4729-494c-a5aa-a75885bab843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:54.963596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db3c2a-4729-494c-a5aa-a75885bab843 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:54.979315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9718d-f920-4e4d-ab3c-bd0f8e6b2f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:54.979944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9718d-f920-4e4d-ab3c-bd0f8e6b2f34 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:54.980222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9718d-f920-4e4d-ab3c-bd0f8e6b2f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:54.980493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9718d-f920-4e4d-ab3c-bd0f8e6b2f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:54.993985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9718d-f920-4e4d-ab3c-bd0f8e6b2f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:54.995455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9718d-f920-4e4d-ab3c-bd0f8e6b2f34 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:55.014922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 838/1685] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:55.187246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93840d8-1af3-4f63-839e-0c97ce888321 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:55.188233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93840d8-1af3-4f63-839e-0c97ce888321 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:55.189584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93840d8-1af3-4f63-839e-0c97ce888321 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:55.192092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 848/1686] 10.222.0.42 () {54 vars in 912 bytes} [Sat Sep 17 03:48:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:48:55.197428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3202-32fb-43d8-b056-84e2b046bcf7 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:55.197714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3202-32fb-43d8-b056-84e2b046bcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:55.198015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3202-32fb-43d8-b056-84e2b046bcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:55.232205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-f74f3202-32fb-43d8-b056-84e2b046bcf7 None None] MFA Rules not processed for user `6336f0c469b84db78a90cf8435c7dbe4`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:55.233423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f74f3202-32fb-43d8-b056-84e2b046bcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:55.256580 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 839/1687] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:48:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:55.622913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93840d8-1af3-4f63-839e-0c97ce888321 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:55.644623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93840d8-1af3-4f63-839e-0c97ce888321 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:55.654047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83570338-a7ce-4e2f-8a97-86e8fa139b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:55.656950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83570338-a7ce-4e2f-8a97-86e8fa139b1d service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:55.657547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83570338-a7ce-4e2f-8a97-86e8fa139b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:55.658087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83570338-a7ce-4e2f-8a97-86e8fa139b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:55.674492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83570338-a7ce-4e2f-8a97-86e8fa139b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:55.676154 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83570338-a7ce-4e2f-8a97-86e8fa139b1d service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:55.694166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 849/1688] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:56.313075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3202-32fb-43d8-b056-84e2b046bcf7 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:56.314567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f74f3202-32fb-43d8-b056-84e2b046bcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:56.484860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c3d63-3213-428c-8b03-c381f505c48e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:56.485173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c3d63-3213-428c-8b03-c381f505c48e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:56.485969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c3d63-3213-428c-8b03-c381f505c48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:56.512725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3202-32fb-43d8-b056-84e2b046bcf7 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:56.525553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e658000-64b8-4315-816a-a234e2a86f19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ec5c9c71d14762a4869b7d66e87ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:56.528043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e658000-64b8-4315-816a-a234e2a86f19 service placement] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:56.528043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e658000-64b8-4315-816a-a234e2a86f19 service placement] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:56.528043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e658000-64b8-4315-816a-a234e2a86f19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:56.529714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8e658000-64b8-4315-816a-a234e2a86f19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:56.563952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-111c3d63-3213-428c-8b03-c381f505c48e None None] MFA Rules not processed for user `26995363963a4a52acc3f57ce1575e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:56.565256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-111c3d63-3213-428c-8b03-c381f505c48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:56.573145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e658000-64b8-4315-816a-a234e2a86f19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:56.576280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e658000-64b8-4315-816a-a234e2a86f19 service placement] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:56.585567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:48:56.595614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 840/1689] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:56.639037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 850/1690] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:57.531576 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e658000-64b8-4315-816a-a234e2a86f19 service placement] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:48:57.561160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e658000-64b8-4315-816a-a234e2a86f19 service placement] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:48:57.594220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bd965-dcd3-41ab-b83b-01aa919e741d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:48:57.598319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bd965-dcd3-41ab-b83b-01aa919e741d service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:57.600117 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bd965-dcd3-41ab-b83b-01aa919e741d service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:57.601356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bd965-dcd3-41ab-b83b-01aa919e741d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:57.800731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bd965-dcd3-41ab-b83b-01aa919e741d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:48:57.808270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bd965-dcd3-41ab-b83b-01aa919e741d service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:48:57.862578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 841/1691] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:48:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:48:57.908865 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bca8ca-e920-40e3-bbc5-d951fb20a750 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:57.909873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bca8ca-e920-40e3-bbc5-d951fb20a750 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:57.910315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bca8ca-e920-40e3-bbc5-d951fb20a750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:57.957491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-30bca8ca-e920-40e3-bbc5-d951fb20a750 None None] MFA Rules not processed for user `2723e5c74bde49f485099b615b36997a`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:57.959375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-30bca8ca-e920-40e3-bbc5-d951fb20a750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:57.980750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 851/1692] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:48:59.642181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad295752-eca0-417b-a2df-48f0cb9e116a None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:48:59.643057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad295752-eca0-417b-a2df-48f0cb9e116a None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:48:59.643471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad295752-eca0-417b-a2df-48f0cb9e116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:48:59.729639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ad295752-eca0-417b-a2df-48f0cb9e116a None None] MFA Rules not processed for user `fafa609d94334db6907b401e66cb3448`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:48:59.732869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ad295752-eca0-417b-a2df-48f0cb9e116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:48:59.752496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 842/1693] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:48:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:01.124777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bca8ca-e920-40e3-bbc5-d951fb20a750 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:01.140990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bca8ca-e920-40e3-bbc5-d951fb20a750 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:01.147301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30817b-364f-4d4d-b8b2-2699d5e4b775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:01.147922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30817b-364f-4d4d-b8b2-2699d5e4b775 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:01.148203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30817b-364f-4d4d-b8b2-2699d5e4b775 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:01.148637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30817b-364f-4d4d-b8b2-2699d5e4b775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:01.179744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30817b-364f-4d4d-b8b2-2699d5e4b775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:01.181272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30817b-364f-4d4d-b8b2-2699d5e4b775 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:01.205973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 852/1694] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:02.652826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad295752-eca0-417b-a2df-48f0cb9e116a None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:02.658928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ad295752-eca0-417b-a2df-48f0cb9e116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:02.734476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad295752-eca0-417b-a2df-48f0cb9e116a None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:02.739618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:02.740247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:02.740520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:02.740787 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] PATH_INFO: `/v3/users/8af07cadb5c548978c00240a5b3e2a08` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:02.741964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af07cadb5c548978c00240a5b3e2a08)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:02.743913 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:03.015304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af07cadb5c548978c00240a5b3e2a08'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:03.097264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af07cadb5c548978c00240a5b3e2a08'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:03.101194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af07cadb5c548978c00240a5b3e2a08'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:03.107563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af07cadb5c548978c00240a5b3e2a08'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:03.120939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af07cadb5c548978c00240a5b3e2a08'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:03.127379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 843/1695] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:02 2022] DELETE /identity/v3/users/8af07cadb5c548978c00240a5b3e2a08 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:03.131914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45527c92-6c8e-4555-bbcf-afd73c68add5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:03.132367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45527c92-6c8e-4555-bbcf-afd73c68add5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:03.132452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45527c92-6c8e-4555-bbcf-afd73c68add5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:03.151230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-45527c92-6c8e-4555-bbcf-afd73c68add5 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:03.152423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-45527c92-6c8e-4555-bbcf-afd73c68add5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:03.220059 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 853/1696] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:03.229329 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:03.230830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:03.244911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:03.289829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91596fdb-3eb6-4d50-8fcb-65759c5f26df admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:03.293339 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12644e39-bbe2-449d-9871-fdad5002b504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:03.294009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12644e39-bbe2-449d-9871-fdad5002b504 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:03.294288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12644e39-bbe2-449d-9871-fdad5002b504 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:03.294557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12644e39-bbe2-449d-9871-fdad5002b504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:03.303493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12644e39-bbe2-449d-9871-fdad5002b504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:03.304905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12644e39-bbe2-449d-9871-fdad5002b504 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:03.314829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 844/1697] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:03.745002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45527c92-6c8e-4555-bbcf-afd73c68add5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:03.746729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-45527c92-6c8e-4555-bbcf-afd73c68add5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:03.762371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45527c92-6c8e-4555-bbcf-afd73c68add5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:03.769221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:03.770001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:03.770207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:03.770478 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] PATH_INFO: `/v3/projects/12eeba70d923445b901341ebcf58a05e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:03.771783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12eeba70d923445b901341ebcf58a05e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:03.773677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:03.891467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12eeba70d923445b901341ebcf58a05e', 'request_id': 'req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:03.897840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12eeba70d923445b901341ebcf58a05e', 'request_id': 'req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:03.972813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12eeba70d923445b901341ebcf58a05e', 'request_id': 'req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:03.980225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:03.985568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] The token cache is being invalidate because project 12eeba70d923445b901341ebcf58a05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:03.985900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:03.986663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:03.991118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 854/1698] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:03 2022] DELETE /identity/v3/projects/12eeba70d923445b901341ebcf58a05e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:03.992302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12644e39-bbe2-449d-9871-fdad5002b504 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:04.003133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12644e39-bbe2-449d-9871-fdad5002b504 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:04.081013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:04.081679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:04.082005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:04.082428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] PATH_INFO: `/v3/users/759f1c920f124a1eb23d2168903b518c` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:04.083588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] RBAC: Authorizing `identity:delete_user(user_id=759f1c920f124a1eb23d2168903b518c)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:04.085757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:04.430966 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '759f1c920f124a1eb23d2168903b518c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:04.544693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '759f1c920f124a1eb23d2168903b518c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:04.545372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '759f1c920f124a1eb23d2168903b518c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:04.563758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '759f1c920f124a1eb23d2168903b518c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:04.580113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '759f1c920f124a1eb23d2168903b518c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:04.586524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 845/1699] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:03 2022] DELETE /identity/v3/users/759f1c920f124a1eb23d2168903b518c => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:05.189711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:05.192190 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:05.230253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be39d1-b5d3-43b3-b986-5f7090e9bb35 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:05.321061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:05.321061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:05.321061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:05.321061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] PATH_INFO: `/v3/projects/b2b0cb3bae844af681c7f6146733ba6f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:05.324337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b0cb3bae844af681c7f6146733ba6f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:05.331381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:05.333493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:05.335161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:05.363171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a6c2a-e584-4794-af03-2100627ee38d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:05.376407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:05.377493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:05.378046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:05.378727 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] PATH_INFO: `/v3/users/b038a963f86849a5b755b24f4bf202f5` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:05.384247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b038a963f86849a5b755b24f4bf202f5)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:05.386817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:05.497968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b0cb3bae844af681c7f6146733ba6f', 'request_id': 'req-0faf691b-b3a5-4a52-bde4-3e8cd191671e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:05.505002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b0cb3bae844af681c7f6146733ba6f', 'request_id': 'req-0faf691b-b3a5-4a52-bde4-3e8cd191671e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:05.567456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:05.605943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b0cb3bae844af681c7f6146733ba6f', 'request_id': 'req-0faf691b-b3a5-4a52-bde4-3e8cd191671e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:05.618571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] The token cache is being invalidate because project b2b0cb3bae844af681c7f6146733ba6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:05.618987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:05.619685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0faf691b-b3a5-4a52-bde4-3e8cd191671e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:05.624709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 855/1700] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:05 2022] DELETE /identity/v3/projects/b2b0cb3bae844af681c7f6146733ba6f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:05.626265 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b85f8a-cfa8-4bea-abab-9d6dbd400782 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:05.626550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b85f8a-cfa8-4bea-abab-9d6dbd400782 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:05.626907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b85f8a-cfa8-4bea-abab-9d6dbd400782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:05.650875 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-80b85f8a-cfa8-4bea-abab-9d6dbd400782 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:05.652161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-80b85f8a-cfa8-4bea-abab-9d6dbd400782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:05.653506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b038a963f86849a5b755b24f4bf202f5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:05.721184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b038a963f86849a5b755b24f4bf202f5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:05.721884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 856/1701] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:05.721992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b038a963f86849a5b755b24f4bf202f5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:05.734385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b038a963f86849a5b755b24f4bf202f5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:05.742652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad8647-ef68-4be3-aa7d-853794055247 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:05.742652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad8647-ef68-4be3-aa7d-853794055247 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:05.742940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad8647-ef68-4be3-aa7d-853794055247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:05.757413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-01a76f30-35dd-4732-82a0-f58ac38bd98b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b038a963f86849a5b755b24f4bf202f5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:05.768674 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 846/1702] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:05 2022] DELETE /identity/v3/users/b038a963f86849a5b755b24f4bf202f5 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:05.774217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182aea9-50b4-435e-9389-c235dda0e916 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:05.776683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182aea9-50b4-435e-9389-c235dda0e916 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:05.776969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182aea9-50b4-435e-9389-c235dda0e916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:05.778592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-aaad8647-ef68-4be3-aa7d-853794055247 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:05.779819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-aaad8647-ef68-4be3-aa7d-853794055247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:05.814063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b182aea9-50b4-435e-9389-c235dda0e916 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:05.815060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b182aea9-50b4-435e-9389-c235dda0e916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:05.864729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 857/1703] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:05.869939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad8647-ef68-4be3-aa7d-853794055247 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:05.887024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad8647-ef68-4be3-aa7d-853794055247 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:05.891474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3696c-8405-4641-8da3-2e921bb7acbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:05.892585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3696c-8405-4641-8da3-2e921bb7acbf admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:05.892801 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3696c-8405-4641-8da3-2e921bb7acbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:05.893091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3696c-8405-4641-8da3-2e921bb7acbf admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:05.893811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3696c-8405-4641-8da3-2e921bb7acbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:05.894127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3696c-8405-4641-8da3-2e921bb7acbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:05.896325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3696c-8405-4641-8da3-2e921bb7acbf admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:05.915235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 858/1704] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:49:05.917616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3696c-8405-4641-8da3-2e921bb7acbf admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:05.941235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3696c-8405-4641-8da3-2e921bb7acbf admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:05.951271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020d357-5915-45a3-af3f-14e7937b34b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:05.952098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020d357-5915-45a3-af3f-14e7937b34b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:05.953082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020d357-5915-45a3-af3f-14e7937b34b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:05.953943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020d357-5915-45a3-af3f-14e7937b34b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:05.965550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020d357-5915-45a3-af3f-14e7937b34b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:05.973769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020d357-5915-45a3-af3f-14e7937b34b4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:05.995662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 847/1705] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:06.005868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d37ce-7a61-4c07-b889-eee112b1d766 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:06.006199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d37ce-7a61-4c07-b889-eee112b1d766 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:06.006477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d37ce-7a61-4c07-b889-eee112b1d766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:06.087861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-fc8d37ce-7a61-4c07-b889-eee112b1d766 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:06.089466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-fc8d37ce-7a61-4c07-b889-eee112b1d766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:06.193633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 859/1706] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:05 2022] POST /identity/v3/projects => generated 367 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:06.195188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020d357-5915-45a3-af3f-14e7937b34b4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:06.201312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d020d357-5915-45a3-af3f-14e7937b34b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:06.225763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 848/1707] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:06.253400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d37ce-7a61-4c07-b889-eee112b1d766 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:06.278671 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d37ce-7a61-4c07-b889-eee112b1d766 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:06.386899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020d357-5915-45a3-af3f-14e7937b34b4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:06.395173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65a394-9976-4b74-b1c7-c88456ee9c28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:06.396137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65a394-9976-4b74-b1c7-c88456ee9c28 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:06.396629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65a394-9976-4b74-b1c7-c88456ee9c28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:06.397107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65a394-9976-4b74-b1c7-c88456ee9c28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:06.475893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d079fb-0a6f-4451-adb3-18b32ae7d084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:06.477143 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d079fb-0a6f-4451-adb3-18b32ae7d084 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:06.484313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d079fb-0a6f-4451-adb3-18b32ae7d084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:06.484313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d079fb-0a6f-4451-adb3-18b32ae7d084 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:06.484313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d079fb-0a6f-4451-adb3-18b32ae7d084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:06.487284 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d079fb-0a6f-4451-adb3-18b32ae7d084 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:06.604042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65a394-9976-4b74-b1c7-c88456ee9c28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:06.609661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65a394-9976-4b74-b1c7-c88456ee9c28 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:06.622991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 860/1708] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:06.654191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 849/1709] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:06 2022] POST /identity/v3/users => generated 284 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:06.655576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65a394-9976-4b74-b1c7-c88456ee9c28 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:06.671925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65a394-9976-4b74-b1c7-c88456ee9c28 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:06.677133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b32e2-6200-41b0-9c21-a8744447e593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:06.677791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b32e2-6200-41b0-9c21-a8744447e593 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:06.678077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b32e2-6200-41b0-9c21-a8744447e593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:06.678370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b32e2-6200-41b0-9c21-a8744447e593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:06.679284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b32e2-6200-41b0-9c21-a8744447e593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:06.680970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b32e2-6200-41b0-9c21-a8744447e593 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:06.716325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 861/1710] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:06.717775 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d079fb-0a6f-4451-adb3-18b32ae7d084 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:06.733081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d079fb-0a6f-4451-adb3-18b32ae7d084 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:06.747963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734869a6-7616-4c53-bbb0-c7c57cf7ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:06.748604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734869a6-7616-4c53-bbb0-c7c57cf7ba76 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:06.748972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734869a6-7616-4c53-bbb0-c7c57cf7ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:06.750323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734869a6-7616-4c53-bbb0-c7c57cf7ba76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:06.751899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734869a6-7616-4c53-bbb0-c7c57cf7ba76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:06.753849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734869a6-7616-4c53-bbb0-c7c57cf7ba76 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:06.774940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 850/1711] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:06.778754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b32e2-6200-41b0-9c21-a8744447e593 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:06.812848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734869a6-7616-4c53-bbb0-c7c57cf7ba76 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:06.821771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b32e2-6200-41b0-9c21-a8744447e593 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:06.835418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-734869a6-7616-4c53-bbb0-c7c57cf7ba76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:06.835661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:06.851112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d1079-ed07-46be-a688-1f5746f13e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:06.852057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d1079-ed07-46be-a688-1f5746f13e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:06.852543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d1079-ed07-46be-a688-1f5746f13e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:06.853051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d1079-ed07-46be-a688-1f5746f13e48 admin admin] PATH_INFO: `/v3/projects/0193b7b86e8c452ca48e414b2d8e722f/users/c8c2820c6752491fab7226f3f09e878d/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:06.894104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d1079-ed07-46be-a688-1f5746f13e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0193b7b86e8c452ca48e414b2d8e722f, user_id=c8c2820c6752491fab7226f3f09e878d, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:06.903589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d1079-ed07-46be-a688-1f5746f13e48 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:07.046551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734869a6-7616-4c53-bbb0-c7c57cf7ba76 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:07.067483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 862/1712] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:06 2022] PUT /identity/v3/projects/0193b7b86e8c452ca48e414b2d8e722f/users/c8c2820c6752491fab7226f3f09e878d/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:07.067483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede249fe-75ff-4284-bf77-e8df728d587e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:07.067483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede249fe-75ff-4284-bf77-e8df728d587e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:07.067483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede249fe-75ff-4284-bf77-e8df728d587e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:07.119551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ede249fe-75ff-4284-bf77-e8df728d587e None None] MFA Rules not processed for user `6336f0c469b84db78a90cf8435c7dbe4`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:07.122893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ede249fe-75ff-4284-bf77-e8df728d587e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:07.293636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa7d02-255f-4c18-9d59-8ed705a57d69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:07.294396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa7d02-255f-4c18-9d59-8ed705a57d69 service swift] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:07.294742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa7d02-255f-4c18-9d59-8ed705a57d69 service swift] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:07.295274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa7d02-255f-4c18-9d59-8ed705a57d69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:07.360237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 863/1713] 10.222.0.18 () {58 vars in 1015 bytes} [Sat Sep 17 03:49:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:07.360237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fc01b-ec5e-4a02-b356-d4b4271916ec None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:07.360237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fc01b-ec5e-4a02-b356-d4b4271916ec None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:07.360237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fc01b-ec5e-4a02-b356-d4b4271916ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:07.399711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa7d02-255f-4c18-9d59-8ed705a57d69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:07.440856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa7d02-255f-4c18-9d59-8ed705a57d69 service swift] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:07.440856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 851/1714] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:49:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:49:07.443616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0f3fc01b-ec5e-4a02-b356-d4b4271916ec None None] MFA Rules not processed for user `c8c2820c6752491fab7226f3f09e878d`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:07.443616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0f3fc01b-ec5e-4a02-b356-d4b4271916ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:07.457344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa7d02-255f-4c18-9d59-8ed705a57d69 service swift] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:07.459066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-eafa7d02-255f-4c18-9d59-8ed705a57d69 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:07.479609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa7d02-255f-4c18-9d59-8ed705a57d69 service swift] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:07.491614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d93c3-50e1-4940-86e7-2decd48c72b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:07.494520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d93c3-50e1-4940-86e7-2decd48c72b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:07.496209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d93c3-50e1-4940-86e7-2decd48c72b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:07.496614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d93c3-50e1-4940-86e7-2decd48c72b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:07.545120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d93c3-50e1-4940-86e7-2decd48c72b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:07.551196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d93c3-50e1-4940-86e7-2decd48c72b2 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:07.579292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 852/1715] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:07.654307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 864/1716] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:07.657519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d93c3-50e1-4940-86e7-2decd48c72b2 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:07.941130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d93c3-50e1-4940-86e7-2decd48c72b2 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:07.976529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ad032-8e6a-49db-95f1-5a10b16302bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:07.981149 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ad032-8e6a-49db-95f1-5a10b16302bf admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:07.982821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ad032-8e6a-49db-95f1-5a10b16302bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:07.987001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ad032-8e6a-49db-95f1-5a10b16302bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:07.988526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ad032-8e6a-49db-95f1-5a10b16302bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:08.000904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ad032-8e6a-49db-95f1-5a10b16302bf admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:08.237026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 853/1717] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:07 2022] POST /identity/v3/projects => generated 367 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:08.241981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3fc01b-ec5e-4a02-b356-d4b4271916ec None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:08.257617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3fc01b-ec5e-4a02-b356-d4b4271916ec None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:08.448071 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85394ae-8cc8-41c0-a20f-3e56e80dc249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:08.449033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85394ae-8cc8-41c0-a20f-3e56e80dc249 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:08.449453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85394ae-8cc8-41c0-a20f-3e56e80dc249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:08.449892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85394ae-8cc8-41c0-a20f-3e56e80dc249 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:08.470476 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85394ae-8cc8-41c0-a20f-3e56e80dc249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:08.474984 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85394ae-8cc8-41c0-a20f-3e56e80dc249 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:08.637285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 865/1718] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:08 2022] POST /identity/v3/users => generated 290 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:08.647656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ad032-8e6a-49db-95f1-5a10b16302bf admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:08.659087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85394ae-8cc8-41c0-a20f-3e56e80dc249 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:08.661217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a85394ae-8cc8-41c0-a20f-3e56e80dc249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:08.663161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ad032-8e6a-49db-95f1-5a10b16302bf admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:08.675915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5aad7-1386-4e84-a330-98533bd7e878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:08.676526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5aad7-1386-4e84-a330-98533bd7e878 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:08.676807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5aad7-1386-4e84-a330-98533bd7e878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:08.677098 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5aad7-1386-4e84-a330-98533bd7e878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:08.677910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5aad7-1386-4e84-a330-98533bd7e878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:08.682002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5aad7-1386-4e84-a330-98533bd7e878 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:08.698433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85394ae-8cc8-41c0-a20f-3e56e80dc249 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:08.699371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 854/1719] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:08.704705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5aad7-1386-4e84-a330-98533bd7e878 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:08.721588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5aad7-1386-4e84-a330-98533bd7e878 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:08.973251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:08.973251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:08.973251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:08.973251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] PATH_INFO: `/v3/projects/3b73849a79164336aa88fbcda196c161` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:08.974686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b73849a79164336aa88fbcda196c161)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:08.975096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94a4b9-b1dd-438b-8516-ab71ac9b1d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:08.975204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:08.975297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94a4b9-b1dd-438b-8516-ab71ac9b1d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:08.975297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94a4b9-b1dd-438b-8516-ab71ac9b1d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:08.975297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94a4b9-b1dd-438b-8516-ab71ac9b1d15 admin admin] PATH_INFO: `/v3/projects/fbcb64092c1d45f98a9d266d81b7a53e/users/7ff3576cb5be46f48c55025d35053d91/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:08.975297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94a4b9-b1dd-438b-8516-ab71ac9b1d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbcb64092c1d45f98a9d266d81b7a53e, user_id=7ff3576cb5be46f48c55025d35053d91, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:08.975297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94a4b9-b1dd-438b-8516-ab71ac9b1d15 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:09.184953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 855/1720] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:08 2022] PUT /identity/v3/projects/fbcb64092c1d45f98a9d266d81b7a53e/users/7ff3576cb5be46f48c55025d35053d91/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:09.189291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94a4b9-b1dd-438b-8516-ab71ac9b1d15 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:09.200808 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b73849a79164336aa88fbcda196c161', 'request_id': 'req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:09.214348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b73849a79164336aa88fbcda196c161', 'request_id': 'req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:09.263523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94a4b9-b1dd-438b-8516-ab71ac9b1d15 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:09.319990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc5e2b-23e6-4ef5-bda1-94da3dda7e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:09.321008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc5e2b-23e6-4ef5-bda1-94da3dda7e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.321456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc5e2b-23e6-4ef5-bda1-94da3dda7e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.321657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc5e2b-23e6-4ef5-bda1-94da3dda7e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.322358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc5e2b-23e6-4ef5-bda1-94da3dda7e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:09.324734 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc5e2b-23e6-4ef5-bda1-94da3dda7e97 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:09.338374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 856/1721] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:09.340012 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc5e2b-23e6-4ef5-bda1-94da3dda7e97 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:09.350124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b73849a79164336aa88fbcda196c161', 'request_id': 'req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:09.357568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc5e2b-23e6-4ef5-bda1-94da3dda7e97 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:09.363778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] The token cache is being invalidate because project 3b73849a79164336aa88fbcda196c161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:09.364231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:09.365169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fbc08fb-09e2-4fa1-b0b0-915f8a99e579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:09.367147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 866/1722] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:08 2022] DELETE /identity/v3/projects/3b73849a79164336aa88fbcda196c161 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:09.373700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84e01e-eed1-44b9-af24-179e64f5d774 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.375740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84e01e-eed1-44b9-af24-179e64f5d774 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.375865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c25d9f-671a-4c51-a2a8-947da7c6ff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:09.375982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84e01e-eed1-44b9-af24-179e64f5d774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.376073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c25d9f-671a-4c51-a2a8-947da7c6ff7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.376073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c25d9f-671a-4c51-a2a8-947da7c6ff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.376245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c25d9f-671a-4c51-a2a8-947da7c6ff7d admin admin] PATH_INFO: `/v3/projects/fbcb64092c1d45f98a9d266d81b7a53e/users/7ff3576cb5be46f48c55025d35053d91/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.378375 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c25d9f-671a-4c51-a2a8-947da7c6ff7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbcb64092c1d45f98a9d266d81b7a53e, user_id=7ff3576cb5be46f48c55025d35053d91, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:09.381048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c25d9f-671a-4c51-a2a8-947da7c6ff7d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:09.396924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 857/1723] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:09 2022] PUT /identity/v3/projects/fbcb64092c1d45f98a9d266d81b7a53e/users/7ff3576cb5be46f48c55025d35053d91/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:09.397968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-df84e01e-eed1-44b9-af24-179e64f5d774 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:09.399453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-df84e01e-eed1-44b9-af24-179e64f5d774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:09.407512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad3625-444f-4d07-b3ba-25e8d0cdb380 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.407611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad3625-444f-4d07-b3ba-25e8d0cdb380 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.407859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad3625-444f-4d07-b3ba-25e8d0cdb380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.460362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-8aad3625-444f-4d07-b3ba-25e8d0cdb380 None None] MFA Rules not processed for user `7ff3576cb5be46f48c55025d35053d91`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:09.462445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8aad3625-444f-4d07-b3ba-25e8d0cdb380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:09.509092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 867/1724] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:09.517697 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb5079-f708-4e7a-9c01-86baefe16b3a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.518062 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb5079-f708-4e7a-9c01-86baefe16b3a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.518935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb5079-f708-4e7a-9c01-86baefe16b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.521245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:09.563551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c3eb5079-f708-4e7a-9c01-86baefe16b3a None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:09.567996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c3eb5079-f708-4e7a-9c01-86baefe16b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:09.586875 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 868/1725] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:09.588441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 858/1726] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:09.591836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb5079-f708-4e7a-9c01-86baefe16b3a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:09.602113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb5079-f708-4e7a-9c01-86baefe16b3a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:09.607416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef9b48-6ebf-48a5-a792-3ba10ad1c7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:09.608087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef9b48-6ebf-48a5-a792-3ba10ad1c7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.608705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef9b48-6ebf-48a5-a792-3ba10ad1c7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.609027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef9b48-6ebf-48a5-a792-3ba10ad1c7b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.609903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad3625-444f-4d07-b3ba-25e8d0cdb380 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:09.610242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef9b48-6ebf-48a5-a792-3ba10ad1c7b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:09.610658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef9b48-6ebf-48a5-a792-3ba10ad1c7b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:09.611339 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8aad3625-444f-4d07-b3ba-25e8d0cdb380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:09.612782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef9b48-6ebf-48a5-a792-3ba10ad1c7b2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:09.634792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 869/1727] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:49:09.647863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef9b48-6ebf-48a5-a792-3ba10ad1c7b2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:09.664252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef9b48-6ebf-48a5-a792-3ba10ad1c7b2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:09.671810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26707280-ca68-4b30-98f1-009a4fd9dd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:09.672413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707280-ca68-4b30-98f1-009a4fd9dd2a admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.672758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707280-ca68-4b30-98f1-009a4fd9dd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.673136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707280-ca68-4b30-98f1-009a4fd9dd2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.673881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26707280-ca68-4b30-98f1-009a4fd9dd2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:09.675862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26707280-ca68-4b30-98f1-009a4fd9dd2a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:09.691919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad3625-444f-4d07-b3ba-25e8d0cdb380 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:09.696796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea333fe-6aa5-4a6f-a7a5-aa712d1b464e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:09.697642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea333fe-6aa5-4a6f-a7a5-aa712d1b464e service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.706358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea333fe-6aa5-4a6f-a7a5-aa712d1b464e service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.706358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea333fe-6aa5-4a6f-a7a5-aa712d1b464e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.708852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea333fe-6aa5-4a6f-a7a5-aa712d1b464e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:09.710339 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea333fe-6aa5-4a6f-a7a5-aa712d1b464e service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:09.721925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 859/1728] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:09.814704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 870/1729] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:09 2022] POST /identity/v3/projects => generated 405 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:09.816132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea333fe-6aa5-4a6f-a7a5-aa712d1b464e service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:09.827048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea333fe-6aa5-4a6f-a7a5-aa712d1b464e service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:09.891256 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a83b90-e18e-42d9-8467-e346c06a6c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:09.891864 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a83b90-e18e-42d9-8467-e346c06a6c24 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.893789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a83b90-e18e-42d9-8467-e346c06a6c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.893789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a83b90-e18e-42d9-8467-e346c06a6c24 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.893789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a83b90-e18e-42d9-8467-e346c06a6c24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:09.894981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a83b90-e18e-42d9-8467-e346c06a6c24 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:09.910125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515d20a-202f-4f7f-acf4-945ff7cdee51 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.910415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515d20a-202f-4f7f-acf4-945ff7cdee51 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.910771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515d20a-202f-4f7f-acf4-945ff7cdee51 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.912542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 871/1730] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:49:09.931035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65746-ff38-4992-a633-795ebfa73f2d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:09.939586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65746-ff38-4992-a633-795ebfa73f2d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:09.939586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65746-ff38-4992-a633-795ebfa73f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:09.952835 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-06c65746-ff38-4992-a633-795ebfa73f2d None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:09.954114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-06c65746-ff38-4992-a633-795ebfa73f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:10.018142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 860/1731] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:09 2022] POST /identity/v3/users => generated 303 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:10.025413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a83b90-e18e-42d9-8467-e346c06a6c24 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:10.035949 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 872/1732] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:49:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:10.037901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a83b90-e18e-42d9-8467-e346c06a6c24 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:10.042873 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda1d81-fa9f-497a-875d-11f9d71826bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:10.054027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda1d81-fa9f-497a-875d-11f9d71826bd admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:10.054027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda1d81-fa9f-497a-875d-11f9d71826bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:10.054027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda1d81-fa9f-497a-875d-11f9d71826bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:10.054027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda1d81-fa9f-497a-875d-11f9d71826bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:10.054027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda1d81-fa9f-497a-875d-11f9d71826bd admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:10.061755 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65746-ff38-4992-a633-795ebfa73f2d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:10.064187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-06c65746-ff38-4992-a633-795ebfa73f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:10.064629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 861/1733] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:10.068617 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda1d81-fa9f-497a-875d-11f9d71826bd admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:10.082316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda1d81-fa9f-497a-875d-11f9d71826bd admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:10.084187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65746-ff38-4992-a633-795ebfa73f2d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:10.087611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0839a-11c7-4748-8f8b-a50c1de6742e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:10.088292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0839a-11c7-4748-8f8b-a50c1de6742e admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:10.094187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0839a-11c7-4748-8f8b-a50c1de6742e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:10.094187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0839a-11c7-4748-8f8b-a50c1de6742e admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:10.094187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0839a-11c7-4748-8f8b-a50c1de6742e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:10.094489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f696fa-9dbf-4fba-8389-ba868336a295 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:10.094489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f696fa-9dbf-4fba-8389-ba868336a295 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:10.094685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0839a-11c7-4748-8f8b-a50c1de6742e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:10.094772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f696fa-9dbf-4fba-8389-ba868336a295 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:10.094772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f696fa-9dbf-4fba-8389-ba868336a295 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:10.103070 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f696fa-9dbf-4fba-8389-ba868336a295 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:10.103774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 862/1734] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:10.106793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f696fa-9dbf-4fba-8389-ba868336a295 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:10.107899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0839a-11c7-4748-8f8b-a50c1de6742e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:10.118244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 873/1735] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:10.118477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0839a-11c7-4748-8f8b-a50c1de6742e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:10.123241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32ce6b-d548-4267-add9-d67ba88e62fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:10.124264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32ce6b-d548-4267-add9-d67ba88e62fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:10.124420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32ce6b-d548-4267-add9-d67ba88e62fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:10.124697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32ce6b-d548-4267-add9-d67ba88e62fb admin admin] PATH_INFO: `/v3/projects/f2ea2cc7e82d4c4aa531e064dddebc9e/users/5002c768f9fa44cea510851640e315b7/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:10.140522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32ce6b-d548-4267-add9-d67ba88e62fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ea2cc7e82d4c4aa531e064dddebc9e, user_id=5002c768f9fa44cea510851640e315b7, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:10.143252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32ce6b-d548-4267-add9-d67ba88e62fb admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:10.297773 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 863/1736] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:10 2022] PUT /identity/v3/projects/f2ea2cc7e82d4c4aa531e064dddebc9e/users/5002c768f9fa44cea510851640e315b7/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:10.308977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdba7b-ea27-4676-9b77-a2bfc10b7111 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:10.309273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdba7b-ea27-4676-9b77-a2bfc10b7111 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:10.309553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdba7b-ea27-4676-9b77-a2bfc10b7111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:10.338122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-babdba7b-ea27-4676-9b77-a2bfc10b7111 None None] MFA Rules not processed for user `5002c768f9fa44cea510851640e315b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:10.339314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-babdba7b-ea27-4676-9b77-a2bfc10b7111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:10.404565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 874/1737] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:10 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:10.410498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32ce6b-d548-4267-add9-d67ba88e62fb admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:10.472298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32ce6b-d548-4267-add9-d67ba88e62fb admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:10.476871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db8d16-191a-4fb0-997a-a5696bb28671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:10.477473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db8d16-191a-4fb0-997a-a5696bb28671 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:10.477795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db8d16-191a-4fb0-997a-a5696bb28671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:10.478073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db8d16-191a-4fb0-997a-a5696bb28671 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:10.478697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db8d16-191a-4fb0-997a-a5696bb28671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:10.480599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db8d16-191a-4fb0-997a-a5696bb28671 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:10.562826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 864/1738] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:10 2022] POST /identity/v3/projects => generated 403 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:10.566802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdba7b-ea27-4676-9b77-a2bfc10b7111 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:10.577745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdba7b-ea27-4676-9b77-a2bfc10b7111 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:10.647867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986b446-51a5-491c-b7da-72c58380146b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:10.648440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986b446-51a5-491c-b7da-72c58380146b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:10.648721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986b446-51a5-491c-b7da-72c58380146b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:10.648992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986b446-51a5-491c-b7da-72c58380146b admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:10.649679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986b446-51a5-491c-b7da-72c58380146b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:10.651952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986b446-51a5-491c-b7da-72c58380146b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:10.671648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:10.788100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 875/1739] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:10 2022] POST /identity/v3/users => generated 308 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:10.790757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db8d16-191a-4fb0-997a-a5696bb28671 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:10.804576 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db8d16-191a-4fb0-997a-a5696bb28671 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:10.810965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabc844-4656-43e4-9e56-e2ca79e3d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:10.811808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabc844-4656-43e4-9e56-e2ca79e3d65f admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:10.812472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabc844-4656-43e4-9e56-e2ca79e3d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:10.816449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabc844-4656-43e4-9e56-e2ca79e3d65f admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:10.816449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabc844-4656-43e4-9e56-e2ca79e3d65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:10.816879 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabc844-4656-43e4-9e56-e2ca79e3d65f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:10.834765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 865/1740] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:10.840590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986b446-51a5-491c-b7da-72c58380146b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:10.851798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986b446-51a5-491c-b7da-72c58380146b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:10.864556 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a06b6-d43e-46cb-ab63-51aa7a2d8dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:10.865245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a06b6-d43e-46cb-ab63-51aa7a2d8dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:10.865577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a06b6-d43e-46cb-ab63-51aa7a2d8dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:10.865892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a06b6-d43e-46cb-ab63-51aa7a2d8dbe admin admin] PATH_INFO: `/v3/projects/995119ea683f489a92818e529cb388bf/users/0074878bb4ef4abbb343c9863db9c7b1/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:10.883002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a06b6-d43e-46cb-ab63-51aa7a2d8dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=995119ea683f489a92818e529cb388bf, user_id=0074878bb4ef4abbb343c9863db9c7b1, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:10.886684 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a06b6-d43e-46cb-ab63-51aa7a2d8dbe admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:11.020538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 876/1741] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:10 2022] PUT /identity/v3/projects/995119ea683f489a92818e529cb388bf/users/0074878bb4ef4abbb343c9863db9c7b1/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:11.022913 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabc844-4656-43e4-9e56-e2ca79e3d65f admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:11.036037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabc844-4656-43e4-9e56-e2ca79e3d65f admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:11.113191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e66a2b-61a0-49f3-a376-3d4533b29b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:11.113191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e66a2b-61a0-49f3-a376-3d4533b29b69 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:11.113191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e66a2b-61a0-49f3-a376-3d4533b29b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:11.113191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e66a2b-61a0-49f3-a376-3d4533b29b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:11.113488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e66a2b-61a0-49f3-a376-3d4533b29b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:11.123206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e66a2b-61a0-49f3-a376-3d4533b29b69 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:11.136889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 866/1742] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:11.143820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a06b6-d43e-46cb-ab63-51aa7a2d8dbe admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:11.157348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a06b6-d43e-46cb-ab63-51aa7a2d8dbe admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:11.162381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c07db9-1c7d-46ae-ae6e-c7ec3d49ee87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:11.163116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c07db9-1c7d-46ae-ae6e-c7ec3d49ee87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:11.163650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c07db9-1c7d-46ae-ae6e-c7ec3d49ee87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:11.164040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c07db9-1c7d-46ae-ae6e-c7ec3d49ee87 admin admin] PATH_INFO: `/v3/projects/995119ea683f489a92818e529cb388bf/users/0074878bb4ef4abbb343c9863db9c7b1/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:11.166620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c07db9-1c7d-46ae-ae6e-c7ec3d49ee87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=995119ea683f489a92818e529cb388bf, user_id=0074878bb4ef4abbb343c9863db9c7b1, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:11.169530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c07db9-1c7d-46ae-ae6e-c7ec3d49ee87 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:11.187791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 877/1743] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:11 2022] PUT /identity/v3/projects/995119ea683f489a92818e529cb388bf/users/0074878bb4ef4abbb343c9863db9c7b1/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:11.198362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff252e4-7316-4bb2-901c-bd4eb3fe9fc2 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:11.198801 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff252e4-7316-4bb2-901c-bd4eb3fe9fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:11.199287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff252e4-7316-4bb2-901c-bd4eb3fe9fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:11.258231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-3ff252e4-7316-4bb2-901c-bd4eb3fe9fc2 None None] MFA Rules not processed for user `0074878bb4ef4abbb343c9863db9c7b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:11.258231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-3ff252e4-7316-4bb2-901c-bd4eb3fe9fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:11.363125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 867/1744] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:11 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:11.374803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c07db9-1c7d-46ae-ae6e-c7ec3d49ee87 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:11.376794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-43c07db9-1c7d-46ae-ae6e-c7ec3d49ee87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:11.466197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c07db9-1c7d-46ae-ae6e-c7ec3d49ee87 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:11.480460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3e695-8df3-459d-9528-c7cbab08a472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:11.481088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3e695-8df3-459d-9528-c7cbab08a472 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:11.481415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3e695-8df3-459d-9528-c7cbab08a472 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:11.481627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3e695-8df3-459d-9528-c7cbab08a472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:11.493577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3e695-8df3-459d-9528-c7cbab08a472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:11.495204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3e695-8df3-459d-9528-c7cbab08a472 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:11.510197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 878/1745] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:11 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:11.722632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff252e4-7316-4bb2-901c-bd4eb3fe9fc2 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:11.830429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3e695-8df3-459d-9528-c7cbab08a472 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:11.832751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6de3e695-8df3-459d-9528-c7cbab08a472 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:11.847901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff252e4-7316-4bb2-901c-bd4eb3fe9fc2 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:11.857634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:11.858931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:11.861742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:11.862049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] PATH_INFO: `/v3/users/5002c768f9fa44cea510851640e315b7` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:11.867011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5002c768f9fa44cea510851640e315b7)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:11.872651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3e695-8df3-459d-9528-c7cbab08a472 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:11.872839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:11.886357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:11.890429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:11.890825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:11.891561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] PATH_INFO: `/v3/users/55f2fbf563c4435b8dedc99b3102cde3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:11.893753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f2fbf563c4435b8dedc99b3102cde3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:11.894795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:49:11.894795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Sep 17 03:49:11.894795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: Sep 17 03:49:11.901221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:12.176210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5002c768f9fa44cea510851640e315b7'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:12.350074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f2fbf563c4435b8dedc99b3102cde3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:12.371822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5002c768f9fa44cea510851640e315b7'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:12.371822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5002c768f9fa44cea510851640e315b7'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:12.371822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5002c768f9fa44cea510851640e315b7'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:12.390753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f2fbf563c4435b8dedc99b3102cde3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:12.395577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-dea19cdd-bd23-4b5f-a406-93ee793cc25f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5002c768f9fa44cea510851640e315b7'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:12.404692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 868/1746] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:11 2022] DELETE /identity/v3/users/5002c768f9fa44cea510851640e315b7 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:12.409542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d97a1-b425-4826-936b-ae1e17192461 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:12.409855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d97a1-b425-4826-936b-ae1e17192461 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:12.410146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d97a1-b425-4826-936b-ae1e17192461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:12.441128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ea5d97a1-b425-4826-936b-ae1e17192461 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:12.442890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ea5d97a1-b425-4826-936b-ae1e17192461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:12.519401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f2fbf563c4435b8dedc99b3102cde3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:12.520289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f2fbf563c4435b8dedc99b3102cde3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:12.539451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2dbeb7e1-5356-469e-99c4-615a6c57e4ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f2fbf563c4435b8dedc99b3102cde3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:12.550189 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 879/1747] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:11 2022] DELETE /identity/v3/users/55f2fbf563c4435b8dedc99b3102cde3 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:12.554752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18fb8c-d3fd-447e-9813-1d127a54c9d6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:12.555200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18fb8c-d3fd-447e-9813-1d127a54c9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:12.555484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18fb8c-d3fd-447e-9813-1d127a54c9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:12.561542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 869/1748] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:12.567334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d97a1-b425-4826-936b-ae1e17192461 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:12.568747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ea5d97a1-b425-4826-936b-ae1e17192461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:12.582917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-cf18fb8c-d3fd-447e-9813-1d127a54c9d6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:12.584804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cf18fb8c-d3fd-447e-9813-1d127a54c9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:12.656900 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d97a1-b425-4826-936b-ae1e17192461 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:12.657505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 880/1749] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:12.668898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18fb8c-d3fd-447e-9813-1d127a54c9d6 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:12.761616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18fb8c-d3fd-447e-9813-1d127a54c9d6 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:12.767686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd4dfb-fa7a-4cd3-980a-435858c73446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:12.768320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd4dfb-fa7a-4cd3-980a-435858c73446 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:12.769287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd4dfb-fa7a-4cd3-980a-435858c73446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:12.769404 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd4dfb-fa7a-4cd3-980a-435858c73446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:12.778799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe3094-df0d-4026-979d-d1c15bcf0560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:12.779414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3094-df0d-4026-979d-d1c15bcf0560 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:12.779689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3094-df0d-4026-979d-d1c15bcf0560 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:12.779995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3094-df0d-4026-979d-d1c15bcf0560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:12.784270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd4dfb-fa7a-4cd3-980a-435858c73446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:12.785713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd4dfb-fa7a-4cd3-980a-435858c73446 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:12.797942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe3094-df0d-4026-979d-d1c15bcf0560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:12.799825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe3094-df0d-4026-979d-d1c15bcf0560 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:12.804919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 881/1750] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:12.823583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 870/1751] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:13.322393 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:13.385328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:13.587259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd4dfb-fa7a-4cd3-980a-435858c73446 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:13.589113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4dfd4dfb-fa7a-4cd3-980a-435858c73446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:13.631630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd4dfb-fa7a-4cd3-980a-435858c73446 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:13.640249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:13.641258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:13.641797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:13.642273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] PATH_INFO: `/v3/projects/f2ea2cc7e82d4c4aa531e064dddebc9e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:13.644523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2ea2cc7e82d4c4aa531e064dddebc9e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:13.647323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:13.793266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe3094-df0d-4026-979d-d1c15bcf0560 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:13.810950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-cffe3094-df0d-4026-979d-d1c15bcf0560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:13.861676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe3094-df0d-4026-979d-d1c15bcf0560 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:13.862011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2ea2cc7e82d4c4aa531e064dddebc9e', 'request_id': 'req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:13.877954 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2ea2cc7e82d4c4aa531e064dddebc9e', 'request_id': 'req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:13.953213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:13.954144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:13.954422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:13.954688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] PATH_INFO: `/v3/projects/a9f513d1f74f452eba7d4f000bc6f5dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:13.955819 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9f513d1f74f452eba7d4f000bc6f5dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:13.956837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: WARNING py.warnings [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 03:49:13.956837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: warnings.warn(msg) Sep 17 03:49:13.956837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: Sep 17 03:49:13.958121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:13.967053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2ea2cc7e82d4c4aa531e064dddebc9e', 'request_id': 'req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:13.980312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] The token cache is being invalidate because project f2ea2cc7e82d4c4aa531e064dddebc9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:13.980864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:13.981678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:13.983220 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 882/1752] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:13 2022] DELETE /identity/v3/projects/f2ea2cc7e82d4c4aa531e064dddebc9e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:13.987640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:13.989152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:14.002537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9cedf-8f04-4306-ac7c-cdae2ab17bb6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:14.007685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:14.008370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:14.008711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:14.009191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] PATH_INFO: `/v3/users/0074878bb4ef4abbb343c9863db9c7b1` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:14.010803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0074878bb4ef4abbb343c9863db9c7b1)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:14.013844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:14.105571 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9f513d1f74f452eba7d4f000bc6f5dd', 'request_id': 'req-b7c900cf-712a-4275-8394-aeaa2adea79c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.275636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9f513d1f74f452eba7d4f000bc6f5dd', 'request_id': 'req-b7c900cf-712a-4275-8394-aeaa2adea79c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.283105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9f513d1f74f452eba7d4f000bc6f5dd', 'request_id': 'req-b7c900cf-712a-4275-8394-aeaa2adea79c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.302451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] The token cache is being invalidate because project a9f513d1f74f452eba7d4f000bc6f5dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:14.302742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.303412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.307575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 871/1753] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:13 2022] DELETE /identity/v3/projects/a9f513d1f74f452eba7d4f000bc6f5dd => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:14.307575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:14.307575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:14.344129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0074878bb4ef4abbb343c9863db9c7b1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.386789 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0074878bb4ef4abbb343c9863db9c7b1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.399650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c900cf-712a-4275-8394-aeaa2adea79c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:14.405085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:14.408082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:14.408445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:14.408804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] PATH_INFO: `/v3/users/c8c2820c6752491fab7226f3f09e878d` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:14.410168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8c2820c6752491fab7226f3f09e878d)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:14.412525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:14.498956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0074878bb4ef4abbb343c9863db9c7b1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.500296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0074878bb4ef4abbb343c9863db9c7b1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.508156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0074878bb4ef4abbb343c9863db9c7b1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.514391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 883/1754] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:13 2022] DELETE /identity/v3/users/0074878bb4ef4abbb343c9863db9c7b1 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:14.517429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:14.518745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:14.646674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e09942-c822-4762-bd74-2c5d9daccf53 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:14.669231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:14.669888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:14.670174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:14.670665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] PATH_INFO: `/v3/users/60ef0cd7192e426685c82754f4deba8c` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:14.676053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8c2820c6752491fab7226f3f09e878d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.676851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ef0cd7192e426685c82754f4deba8c)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:14.679087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:14.748229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8c2820c6752491fab7226f3f09e878d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.749041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8c2820c6752491fab7226f3f09e878d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.762960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8c2820c6752491fab7226f3f09e878d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.773465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-38c7187c-139e-4381-add1-ec1615d68ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8c2820c6752491fab7226f3f09e878d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.814173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 872/1755] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:14 2022] DELETE /identity/v3/users/c8c2820c6752491fab7226f3f09e878d => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:14.828056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0530b-7eca-4ac1-94e5-b1cea9f8913b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:14.828659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0530b-7eca-4ac1-94e5-b1cea9f8913b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:14.829157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0530b-7eca-4ac1-94e5-b1cea9f8913b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:14.851918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a0b0530b-7eca-4ac1-94e5-b1cea9f8913b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:14.854136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a0b0530b-7eca-4ac1-94e5-b1cea9f8913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:14.947885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 873/1756] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:14.952634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ef0cd7192e426685c82754f4deba8c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:14.965765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0530b-7eca-4ac1-94e5-b1cea9f8913b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:14.970807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a0b0530b-7eca-4ac1-94e5-b1cea9f8913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:14.997976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ef0cd7192e426685c82754f4deba8c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:15.054266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0530b-7eca-4ac1-94e5-b1cea9f8913b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:15.057972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50840204-df8f-4281-a084-daef249307c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:15.058573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50840204-df8f-4281-a084-daef249307c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:15.058899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50840204-df8f-4281-a084-daef249307c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:15.059186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50840204-df8f-4281-a084-daef249307c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:15.071875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50840204-df8f-4281-a084-daef249307c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:15.073532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50840204-df8f-4281-a084-daef249307c5 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:15.096458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 874/1757] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:15.104415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ef0cd7192e426685c82754f4deba8c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:15.105040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ef0cd7192e426685c82754f4deba8c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:15.119434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a63b6bb8-de93-4344-ac07-6d5735daab82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ef0cd7192e426685c82754f4deba8c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:15.119822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50840204-df8f-4281-a084-daef249307c5 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:15.122390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-50840204-df8f-4281-a084-daef249307c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:15.134356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 884/1758] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:14 2022] DELETE /identity/v3/users/60ef0cd7192e426685c82754f4deba8c => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:15.157314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50840204-df8f-4281-a084-daef249307c5 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:15.290481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:15.291971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:15.292626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:15.293035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] PATH_INFO: `/v3/projects/995119ea683f489a92818e529cb388bf` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:15.301972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=995119ea683f489a92818e529cb388bf)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:15.306030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:15.490107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '995119ea683f489a92818e529cb388bf', 'request_id': 'req-112b95af-3448-44ae-8014-f3bb597b3e84'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:15.629705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '995119ea683f489a92818e529cb388bf', 'request_id': 'req-112b95af-3448-44ae-8014-f3bb597b3e84'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:15.637720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '995119ea683f489a92818e529cb388bf', 'request_id': 'req-112b95af-3448-44ae-8014-f3bb597b3e84'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:15.659514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] The token cache is being invalidate because project 995119ea683f489a92818e529cb388bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:15.659837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:15.660511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:15.662245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 875/1759] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:15 2022] DELETE /identity/v3/projects/995119ea683f489a92818e529cb388bf => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:15.667724 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db13c7-484b-483d-b8d8-549f6904f46c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:15.671964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db13c7-484b-483d-b8d8-549f6904f46c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:15.671964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db13c7-484b-483d-b8d8-549f6904f46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:15.691170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-99db13c7-484b-483d-b8d8-549f6904f46c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:15.692440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-99db13c7-484b-483d-b8d8-549f6904f46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:15.758640 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:15.760679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:15.792858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 885/1760] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:15.794474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719f3ce-ac8f-4370-af7b-35b97f68ab70 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:15.794768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719f3ce-ac8f-4370-af7b-35b97f68ab70 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:15.795049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719f3ce-ac8f-4370-af7b-35b97f68ab70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:15.833870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7719f3ce-ac8f-4370-af7b-35b97f68ab70 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:15.835129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7719f3ce-ac8f-4370-af7b-35b97f68ab70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:15.857506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 886/1761] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:15.861791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719f3ce-ac8f-4370-af7b-35b97f68ab70 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:15.875535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719f3ce-ac8f-4370-af7b-35b97f68ab70 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:15.885913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054462c1-164b-4e17-98cd-242baafe6e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:15.886552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054462c1-164b-4e17-98cd-242baafe6e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:15.886836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054462c1-164b-4e17-98cd-242baafe6e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:15.887120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054462c1-164b-4e17-98cd-242baafe6e22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:15.887866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054462c1-164b-4e17-98cd-242baafe6e22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:15.888086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054462c1-164b-4e17-98cd-242baafe6e22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:15.890343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054462c1-164b-4e17-98cd-242baafe6e22 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:15.910034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b95af-3448-44ae-8014-f3bb597b3e84 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:15.914167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 887/1762] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:49:15.918115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:15.918608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054462c1-164b-4e17-98cd-242baafe6e22 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:15.918829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:15.919006 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:15.919368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] PATH_INFO: `/v3/projects/0193b7b86e8c452ca48e414b2d8e722f` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:15.920431 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0193b7b86e8c452ca48e414b2d8e722f)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:15.922432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:15.930447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054462c1-164b-4e17-98cd-242baafe6e22 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:15.935287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916a8cc-a88a-4ef9-90cc-03d007f7700c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:15.937010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916a8cc-a88a-4ef9-90cc-03d007f7700c admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:15.937328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916a8cc-a88a-4ef9-90cc-03d007f7700c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:15.937624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916a8cc-a88a-4ef9-90cc-03d007f7700c admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:15.938525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916a8cc-a88a-4ef9-90cc-03d007f7700c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:15.940494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916a8cc-a88a-4ef9-90cc-03d007f7700c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:16.111941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 888/1763] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:15 2022] POST /identity/v3/projects => generated 371 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:16.112868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916a8cc-a88a-4ef9-90cc-03d007f7700c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:16.117885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6916a8cc-a88a-4ef9-90cc-03d007f7700c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:16.191257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916a8cc-a88a-4ef9-90cc-03d007f7700c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:16.198451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0193b7b86e8c452ca48e414b2d8e722f', 'request_id': 'req-92a14b33-5b6f-4b84-a213-2118e969466d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.260540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:16.261192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:16.261475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:16.261784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] PATH_INFO: `/v3/projects/f1a83565f60e4cc4909033e2ded526c4` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:16.262927 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a83565f60e4cc4909033e2ded526c4)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:16.265424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:16.321744 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0193b7b86e8c452ca48e414b2d8e722f', 'request_id': 'req-92a14b33-5b6f-4b84-a213-2118e969466d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.329808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0193b7b86e8c452ca48e414b2d8e722f', 'request_id': 'req-92a14b33-5b6f-4b84-a213-2118e969466d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.341939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] The token cache is being invalidate because project 0193b7b86e8c452ca48e414b2d8e722f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:16.342153 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.343384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.344861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 876/1764] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:15 2022] DELETE /identity/v3/projects/0193b7b86e8c452ca48e414b2d8e722f => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:16.345629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:16.346946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:16.359087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a14b33-5b6f-4b84-a213-2118e969466d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:16.364043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbead57-c28c-40ea-ba20-6c973f209f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:16.364627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbead57-c28c-40ea-ba20-6c973f209f54 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:16.364904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbead57-c28c-40ea-ba20-6c973f209f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:16.365174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbead57-c28c-40ea-ba20-6c973f209f54 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:16.365848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbead57-c28c-40ea-ba20-6c973f209f54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:16.367752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbead57-c28c-40ea-ba20-6c973f209f54 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:16.473772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a83565f60e4cc4909033e2ded526c4', 'request_id': 'req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.482565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a83565f60e4cc4909033e2ded526c4', 'request_id': 'req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.484017 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 877/1765] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:16 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:16.488000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbead57-c28c-40ea-ba20-6c973f209f54 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:16.489863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0cbead57-c28c-40ea-ba20-6c973f209f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:16.503058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbead57-c28c-40ea-ba20-6c973f209f54 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:16.574768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:16.575510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:16.575852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:16.576231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] PATH_INFO: `/v3/users/7ff3576cb5be46f48c55025d35053d91` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:16.577880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff3576cb5be46f48c55025d35053d91)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:16.580529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:16.583327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:16.618455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a83565f60e4cc4909033e2ded526c4', 'request_id': 'req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.634292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] The token cache is being invalidate because project f1a83565f60e4cc4909033e2ded526c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:16.634584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.635266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.636790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 889/1766] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:16 2022] DELETE /identity/v3/projects/f1a83565f60e4cc4909033e2ded526c4 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:16.637627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:16.638989 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:16.658509 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1f453-cadb-47c7-aa19-9a8f8a22d7f3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:16.663746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98938fb9-234a-40bc-bf79-eab4134eac09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:16.664661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98938fb9-234a-40bc-bf79-eab4134eac09 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:16.665154 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98938fb9-234a-40bc-bf79-eab4134eac09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:16.665639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98938fb9-234a-40bc-bf79-eab4134eac09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:16.666529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98938fb9-234a-40bc-bf79-eab4134eac09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:16.668429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98938fb9-234a-40bc-bf79-eab4134eac09 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:16.685397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 890/1767] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:16.686390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98938fb9-234a-40bc-bf79-eab4134eac09 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:16.690095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-98938fb9-234a-40bc-bf79-eab4134eac09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:16.705246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98938fb9-234a-40bc-bf79-eab4134eac09 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:16.711308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:16.713017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:16.713370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:16.713735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] PATH_INFO: `/v3/users/d0d3b3d878a84a5d8f6e165a7c556499` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:16.715432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d3b3d878a84a5d8f6e165a7c556499)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:16.719210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:16.791104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff3576cb5be46f48c55025d35053d91'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.902446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff3576cb5be46f48c55025d35053d91'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.903057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff3576cb5be46f48c55025d35053d91'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.911210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff3576cb5be46f48c55025d35053d91'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.922765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff3576cb5be46f48c55025d35053d91'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:16.928859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 878/1768] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:16 2022] DELETE /identity/v3/users/7ff3576cb5be46f48c55025d35053d91 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:16.929472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:16.930838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:16.986778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d3b3d878a84a5d8f6e165a7c556499'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:17.032621 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24791696-2184-49e1-b52e-dc3d1eeb8a88 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:17.038407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa3707-882f-4353-8868-ec653e346dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:17.039345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa3707-882f-4353-8868-ec653e346dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:17.039692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa3707-882f-4353-8868-ec653e346dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:17.040328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa3707-882f-4353-8868-ec653e346dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:17.040727 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa3707-882f-4353-8868-ec653e346dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:17.042665 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa3707-882f-4353-8868-ec653e346dd5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:17.054928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d3b3d878a84a5d8f6e165a7c556499'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:17.061469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 879/1769] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:17.068191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa3707-882f-4353-8868-ec653e346dd5 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:17.079530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa3707-882f-4353-8868-ec653e346dd5 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:17.084501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59440d-ca35-4036-bc0a-12ed863ee687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:17.085158 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59440d-ca35-4036-bc0a-12ed863ee687 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:17.085429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59440d-ca35-4036-bc0a-12ed863ee687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:17.085704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59440d-ca35-4036-bc0a-12ed863ee687 admin admin] PATH_INFO: `/v3/projects/798c8bf087f8499b81057bbf2c532cf7/users/67d461a8196745219340ca93abacc849/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:17.104480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59440d-ca35-4036-bc0a-12ed863ee687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798c8bf087f8499b81057bbf2c532cf7, user_id=67d461a8196745219340ca93abacc849, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:17.108116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59440d-ca35-4036-bc0a-12ed863ee687 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:17.250978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d3b3d878a84a5d8f6e165a7c556499'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:17.251958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d3b3d878a84a5d8f6e165a7c556499'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:17.255048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 880/1770] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:17 2022] PUT /identity/v3/projects/798c8bf087f8499b81057bbf2c532cf7/users/67d461a8196745219340ca93abacc849/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:17.262097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d3b3d878a84a5d8f6e165a7c556499'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:17.267492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1c8ab-b316-43be-9795-814fca21f4e7 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:17.268058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1c8ab-b316-43be-9795-814fca21f4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:17.268770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1c8ab-b316-43be-9795-814fca21f4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:17.277640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 891/1771] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:16 2022] DELETE /identity/v3/users/d0d3b3d878a84a5d8f6e165a7c556499 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:17.318456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-85f1c8ab-b316-43be-9795-814fca21f4e7 None None] MFA Rules not processed for user `67d461a8196745219340ca93abacc849`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:17.320479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-85f1c8ab-b316-43be-9795-814fca21f4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:17.418038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 881/1772] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:17.422669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:17.425308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:17.565798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4ee81-8293-4dee-9921-9016ff24e302 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:17.579117 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28431715-873b-49e0-bf6b-311b4099efc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:17.579750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28431715-873b-49e0-bf6b-311b4099efc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:17.580029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28431715-873b-49e0-bf6b-311b4099efc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:17.580297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28431715-873b-49e0-bf6b-311b4099efc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:17.580972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28431715-873b-49e0-bf6b-311b4099efc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:17.582975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28431715-873b-49e0-bf6b-311b4099efc8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:17.703224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 892/1773] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:17 2022] POST /identity/v3/projects => generated 371 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:17.707094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1c8ab-b316-43be-9795-814fca21f4e7 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:17.716748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28431715-873b-49e0-bf6b-311b4099efc8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:17.721563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-28431715-873b-49e0-bf6b-311b4099efc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:17.724008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1c8ab-b316-43be-9795-814fca21f4e7 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:17.874852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e962f-6c79-445c-a951-cb4eae126b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:17.876470 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e962f-6c79-445c-a951-cb4eae126b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:17.876952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e962f-6c79-445c-a951-cb4eae126b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:17.879537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e962f-6c79-445c-a951-cb4eae126b63 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:17.879537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e962f-6c79-445c-a951-cb4eae126b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:17.880740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e962f-6c79-445c-a951-cb4eae126b63 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:17.890820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28431715-873b-49e0-bf6b-311b4099efc8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:17.896318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:17.896832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:17.897121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:17.897392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] PATH_INFO: `/v3/projects/fbcb64092c1d45f98a9d266d81b7a53e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:17.898872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbcb64092c1d45f98a9d266d81b7a53e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:17.900775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:17.905655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:17.977352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 882/1774] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:17 2022] POST /identity/v3/users => generated 292 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:17.981431 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e962f-6c79-445c-a951-cb4eae126b63 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:17.999587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e962f-6c79-445c-a951-cb4eae126b63 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:18.007514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c5eec-f072-40b0-99b2-392ac7a38dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:18.008196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c5eec-f072-40b0-99b2-392ac7a38dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:18.008482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c5eec-f072-40b0-99b2-392ac7a38dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:18.008758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c5eec-f072-40b0-99b2-392ac7a38dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:18.009414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c5eec-f072-40b0-99b2-392ac7a38dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:18.011370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c5eec-f072-40b0-99b2-392ac7a38dd9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:18.026411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 883/1775] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:18.033469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c5eec-f072-40b0-99b2-392ac7a38dd9 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:18.041663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c5eec-f072-40b0-99b2-392ac7a38dd9 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:18.046926 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d668755-3bff-463f-ac02-98d7728aebc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:18.047590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d668755-3bff-463f-ac02-98d7728aebc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:18.047865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d668755-3bff-463f-ac02-98d7728aebc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:18.048134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d668755-3bff-463f-ac02-98d7728aebc2 admin admin] PATH_INFO: `/v3/projects/885e698cb07d4e668697bb2740ac17de/users/364756511bc64c5c998655055b7cbbb9/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:18.081552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d668755-3bff-463f-ac02-98d7728aebc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=885e698cb07d4e668697bb2740ac17de, user_id=364756511bc64c5c998655055b7cbbb9, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:18.084545 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d668755-3bff-463f-ac02-98d7728aebc2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:18.227103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 884/1776] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:18 2022] PUT /identity/v3/projects/885e698cb07d4e668697bb2740ac17de/users/364756511bc64c5c998655055b7cbbb9/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:18.231070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d668755-3bff-463f-ac02-98d7728aebc2 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:18.233827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6d668755-3bff-463f-ac02-98d7728aebc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:18.239377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbcb64092c1d45f98a9d266d81b7a53e', 'request_id': 'req-4b371d72-4b77-4d82-a437-0deebd4fa46f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:18.245870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbcb64092c1d45f98a9d266d81b7a53e', 'request_id': 'req-4b371d72-4b77-4d82-a437-0deebd4fa46f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:18.299629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d668755-3bff-463f-ac02-98d7728aebc2 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:18.359675 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:18.360427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:18.360701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:18.361568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] PATH_INFO: `/v3/projects/49f2070fe01b4cd89fd59d3463e9482f` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:18.362738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f2070fe01b4cd89fd59d3463e9482f)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:18.364829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:19.601862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbcb64092c1d45f98a9d266d81b7a53e', 'request_id': 'req-4b371d72-4b77-4d82-a437-0deebd4fa46f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:19.617710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] The token cache is being invalidate because project fbcb64092c1d45f98a9d266d81b7a53e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:19.618035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:19.632942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:19.632942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 893/1777] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:17 2022] DELETE /identity/v3/projects/fbcb64092c1d45f98a9d266d81b7a53e => generated 0 bytes in 1904 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:19.632942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:19.632942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:19.637947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f2070fe01b4cd89fd59d3463e9482f', 'request_id': 'req-fa4980f2-f32b-4a0e-8736-b06943f93f02'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:19.644882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b371d72-4b77-4d82-a437-0deebd4fa46f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:19.716889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d91295-d0c6-4b48-afbc-ab8cf07c909a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:19.717689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d91295-d0c6-4b48-afbc-ab8cf07c909a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:19.718086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d91295-d0c6-4b48-afbc-ab8cf07c909a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:19.718427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d91295-d0c6-4b48-afbc-ab8cf07c909a admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:19.719141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d91295-d0c6-4b48-afbc-ab8cf07c909a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:19.720819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d91295-d0c6-4b48-afbc-ab8cf07c909a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:19.735031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 894/1778] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:19.735031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ef5f1-cbeb-43cb-b442-d08da7d29ecb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:19.735031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ef5f1-cbeb-43cb-b442-d08da7d29ecb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:19.735031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ef5f1-cbeb-43cb-b442-d08da7d29ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:19.754590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-cd9ef5f1-cbeb-43cb-b442-d08da7d29ecb None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:19.755830 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cd9ef5f1-cbeb-43cb-b442-d08da7d29ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:19.764287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 895/1779] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:19.769496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ef5f1-cbeb-43cb-b442-d08da7d29ecb None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:19.780683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ef5f1-cbeb-43cb-b442-d08da7d29ecb None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:19.786202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c80842-2ef5-4f79-8d8d-5126676d392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:19.787225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c80842-2ef5-4f79-8d8d-5126676d392e admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:19.787225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c80842-2ef5-4f79-8d8d-5126676d392e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:19.787446 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c80842-2ef5-4f79-8d8d-5126676d392e admin admin] PATH_INFO: `/v3/projects/885e698cb07d4e668697bb2740ac17de/users/364756511bc64c5c998655055b7cbbb9/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:19.789343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c80842-2ef5-4f79-8d8d-5126676d392e admin admin] RBAC: Authorizing `identity:create_grant(project_id=885e698cb07d4e668697bb2740ac17de, user_id=364756511bc64c5c998655055b7cbbb9, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:19.792434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c80842-2ef5-4f79-8d8d-5126676d392e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:19.825349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 896/1780] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:19 2022] PUT /identity/v3/projects/885e698cb07d4e668697bb2740ac17de/users/364756511bc64c5c998655055b7cbbb9/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:19.826938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbfe60-6304-493e-8568-37a0d29f2c12 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:19.827545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbfe60-6304-493e-8568-37a0d29f2c12 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:19.827723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbfe60-6304-493e-8568-37a0d29f2c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:19.857166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-88dbfe60-6304-493e-8568-37a0d29f2c12 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:19.859818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-88dbfe60-6304-493e-8568-37a0d29f2c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:19.931396 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 897/1781] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:19.934609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eed2eb-dce4-4ef3-976e-7f4802854377 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:19.935414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eed2eb-dce4-4ef3-976e-7f4802854377 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:19.936098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eed2eb-dce4-4ef3-976e-7f4802854377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:19.970477 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f5eed2eb-dce4-4ef3-976e-7f4802854377 None None] MFA Rules not processed for user `364756511bc64c5c998655055b7cbbb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:19.971839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f5eed2eb-dce4-4ef3-976e-7f4802854377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:19.979239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f2070fe01b4cd89fd59d3463e9482f', 'request_id': 'req-fa4980f2-f32b-4a0e-8736-b06943f93f02'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:19.987630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f2070fe01b4cd89fd59d3463e9482f', 'request_id': 'req-fa4980f2-f32b-4a0e-8736-b06943f93f02'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:20.000132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] The token cache is being invalidate because project 49f2070fe01b4cd89fd59d3463e9482f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:20.000394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:20.002184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:20.005035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 885/1782] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:18 2022] DELETE /identity/v3/projects/49f2070fe01b4cd89fd59d3463e9482f => generated 0 bytes in 1777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:20.006669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:20.012726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:20.026489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4980f2-f32b-4a0e-8736-b06943f93f02 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:20.040627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5594475-6084-411a-8426-56a75f48348c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:20.056498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5594475-6084-411a-8426-56a75f48348c admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.056498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5594475-6084-411a-8426-56a75f48348c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.056498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5594475-6084-411a-8426-56a75f48348c admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.056498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5594475-6084-411a-8426-56a75f48348c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:20.056498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5594475-6084-411a-8426-56a75f48348c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:20.056498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5594475-6084-411a-8426-56a75f48348c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:20.057710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 886/1783] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:49:20.058995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f9950-fd30-4612-95a3-dc66029bd1fe None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.059271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f9950-fd30-4612-95a3-dc66029bd1fe None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.059553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f9950-fd30-4612-95a3-dc66029bd1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.065589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 898/1784] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:20.066687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eed2eb-dce4-4ef3-976e-7f4802854377 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:20.085704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6b1f9950-fd30-4612-95a3-dc66029bd1fe None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:20.087334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6b1f9950-fd30-4612-95a3-dc66029bd1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:20.096278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eed2eb-dce4-4ef3-976e-7f4802854377 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:20.106785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 887/1785] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:20.106785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f9950-fd30-4612-95a3-dc66029bd1fe None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:20.106785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6b1f9950-fd30-4612-95a3-dc66029bd1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:20.114182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddec08-5283-4bf8-b452-b45143a73409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:20.114182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddec08-5283-4bf8-b452-b45143a73409 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.114182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddec08-5283-4bf8-b452-b45143a73409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.114182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddec08-5283-4bf8-b452-b45143a73409 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.114479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddec08-5283-4bf8-b452-b45143a73409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:20.114479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddec08-5283-4bf8-b452-b45143a73409 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:20.190076 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f9950-fd30-4612-95a3-dc66029bd1fe None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:20.194129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e05821-60a3-4f72-8d1a-5e75ab6a21f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:20.194719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e05821-60a3-4f72-8d1a-5e75ab6a21f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.194997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e05821-60a3-4f72-8d1a-5e75ab6a21f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.195267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e05821-60a3-4f72-8d1a-5e75ab6a21f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.205595 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e05821-60a3-4f72-8d1a-5e75ab6a21f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:20.207651 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e05821-60a3-4f72-8d1a-5e75ab6a21f1 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:20.223191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 888/1786] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:20.223191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08955bc-31dd-40fa-8af2-6b5a2be929ee None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.223191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08955bc-31dd-40fa-8af2-6b5a2be929ee None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.223191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08955bc-31dd-40fa-8af2-6b5a2be929ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.242736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b08955bc-31dd-40fa-8af2-6b5a2be929ee None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:20.244086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b08955bc-31dd-40fa-8af2-6b5a2be929ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:20.269511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 889/1787] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:20.274820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08955bc-31dd-40fa-8af2-6b5a2be929ee None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:20.289007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08955bc-31dd-40fa-8af2-6b5a2be929ee None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:20.299380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6fe55-a206-4bea-a416-c1ae2f195e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:20.300482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6fe55-a206-4bea-a416-c1ae2f195e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.300905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6fe55-a206-4bea-a416-c1ae2f195e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.301594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6fe55-a206-4bea-a416-c1ae2f195e44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.302545 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6fe55-a206-4bea-a416-c1ae2f195e44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:20.302991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6fe55-a206-4bea-a416-c1ae2f195e44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:20.305053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6fe55-a206-4bea-a416-c1ae2f195e44 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:20.312488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 899/1788] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:20 2022] POST /identity/v3/projects => generated 369 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:20.314149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddec08-5283-4bf8-b452-b45143a73409 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:20.333964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 890/1789] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:49:20.339351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6fe55-a206-4bea-a416-c1ae2f195e44 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:20.353572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6fe55-a206-4bea-a416-c1ae2f195e44 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:20.474334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddec08-5283-4bf8-b452-b45143a73409 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:20.486878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943e43d-b596-47d8-9784-e4213aea1172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:20.488697 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943e43d-b596-47d8-9784-e4213aea1172 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.489050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943e43d-b596-47d8-9784-e4213aea1172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.489261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943e43d-b596-47d8-9784-e4213aea1172 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.490142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943e43d-b596-47d8-9784-e4213aea1172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:20.493502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943e43d-b596-47d8-9784-e4213aea1172 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:20.526157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cf6eb-58a2-4675-aaf6-f31766ad180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:20.526779 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cf6eb-58a2-4675-aaf6-f31766ad180a admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.527057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cf6eb-58a2-4675-aaf6-f31766ad180a admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.527399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cf6eb-58a2-4675-aaf6-f31766ad180a admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.528037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cf6eb-58a2-4675-aaf6-f31766ad180a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:20.531457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cf6eb-58a2-4675-aaf6-f31766ad180a admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:20.701491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 900/1790] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:20 2022] POST /identity/v3/users => generated 285 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:20.710380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 891/1791] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:20 2022] POST /identity/v3/projects => generated 373 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:20.711544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943e43d-b596-47d8-9784-e4213aea1172 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:20.714774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cf6eb-58a2-4675-aaf6-f31766ad180a admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:20.731768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:20.735018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943e43d-b596-47d8-9784-e4213aea1172 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:20.855706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80acaf-cc7a-4b6d-9f05-3be284c2d47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:20.856524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80acaf-cc7a-4b6d-9f05-3be284c2d47a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.856890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80acaf-cc7a-4b6d-9f05-3be284c2d47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.857231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80acaf-cc7a-4b6d-9f05-3be284c2d47a admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.859792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80acaf-cc7a-4b6d-9f05-3be284c2d47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:20.860114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80acaf-cc7a-4b6d-9f05-3be284c2d47a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:20.880803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 901/1792] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:20.886768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80acaf-cc7a-4b6d-9f05-3be284c2d47a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:20.907867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80acaf-cc7a-4b6d-9f05-3be284c2d47a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:20.911288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cf6eb-58a2-4675-aaf6-f31766ad180a admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:20.913505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77069ab-155d-4526-a79a-a46fa85ab1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:20.914266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77069ab-155d-4526-a79a-a46fa85ab1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.914705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77069ab-155d-4526-a79a-a46fa85ab1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.915464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77069ab-155d-4526-a79a-a46fa85ab1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.916273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77069ab-155d-4526-a79a-a46fa85ab1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:20.922733 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3084c-9c40-4de8-bda3-859c7dbce254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:20.922875 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77069ab-155d-4526-a79a-a46fa85ab1d2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:20.923356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3084c-9c40-4de8-bda3-859c7dbce254 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.923743 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3084c-9c40-4de8-bda3-859c7dbce254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.923933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3084c-9c40-4de8-bda3-859c7dbce254 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:20.924567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3084c-9c40-4de8-bda3-859c7dbce254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:20.926820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3084c-9c40-4de8-bda3-859c7dbce254 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:20.937607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 902/1793] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:20.944804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77069ab-155d-4526-a79a-a46fa85ab1d2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:20.962795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77069ab-155d-4526-a79a-a46fa85ab1d2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:20.978710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c73c9-bc88-4eb0-844b-57338e835d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:20.981832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c73c9-bc88-4eb0-844b-57338e835d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:20.981832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c73c9-bc88-4eb0-844b-57338e835d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:20.981832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c73c9-bc88-4eb0-844b-57338e835d5f admin admin] PATH_INFO: `/v3/projects/88e93d0a887445a4a5ebefde211dd186/users/0c5c94811fbc4fef824edd6653963c2b/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.003848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c73c9-bc88-4eb0-844b-57338e835d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e93d0a887445a4a5ebefde211dd186, user_id=0c5c94811fbc4fef824edd6653963c2b, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:21.008063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c73c9-bc88-4eb0-844b-57338e835d5f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:21.106740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 903/1794] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:20 2022] PUT /identity/v3/projects/88e93d0a887445a4a5ebefde211dd186/users/0c5c94811fbc4fef824edd6653963c2b/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:21.109413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea47d-efa5-4654-b000-1e800d9d3466 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.109568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea47d-efa5-4654-b000-1e800d9d3466 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.109869 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea47d-efa5-4654-b000-1e800d9d3466 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.111500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 904/1795] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:49:21.112510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6eaf1-9f79-40bc-babb-a0098eab04bc None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.112998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6eaf1-9f79-40bc-babb-a0098eab04bc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.112998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6eaf1-9f79-40bc-babb-a0098eab04bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.120600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 892/1796] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:20 2022] POST /identity/v3/users => generated 287 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:21.121935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3084c-9c40-4de8-bda3-859c7dbce254 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:21.142631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3084c-9c40-4de8-bda3-859c7dbce254 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:21.154511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2bd6eaf1-9f79-40bc-babb-a0098eab04bc None None] MFA Rules not processed for user `0c5c94811fbc4fef824edd6653963c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:21.156650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2bd6eaf1-9f79-40bc-babb-a0098eab04bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:21.253205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabfa0d-7c69-4b0c-9879-d0ead01aa437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:21.254024 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabfa0d-7c69-4b0c-9879-d0ead01aa437 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.254311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabfa0d-7c69-4b0c-9879-d0ead01aa437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.254674 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabfa0d-7c69-4b0c-9879-d0ead01aa437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.255232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabfa0d-7c69-4b0c-9879-d0ead01aa437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:21.256859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabfa0d-7c69-4b0c-9879-d0ead01aa437 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:21.262887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 905/1797] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:21.265667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab799da8-31d1-4834-9812-710191c5ecc3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.266970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab799da8-31d1-4834-9812-710191c5ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.267360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab799da8-31d1-4834-9812-710191c5ecc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.273867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 893/1798] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:21.273867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabfa0d-7c69-4b0c-9879-d0ead01aa437 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:21.306753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabfa0d-7c69-4b0c-9879-d0ead01aa437 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:21.306753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af6ca9-ca9d-46a4-8bf9-bfe61408c9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:21.309751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af6ca9-ca9d-46a4-8bf9-bfe61408c9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.310211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af6ca9-ca9d-46a4-8bf9-bfe61408c9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.310649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af6ca9-ca9d-46a4-8bf9-bfe61408c9b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.312099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af6ca9-ca9d-46a4-8bf9-bfe61408c9b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:21.316025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af6ca9-ca9d-46a4-8bf9-bfe61408c9b6 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:21.380828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ab799da8-31d1-4834-9812-710191c5ecc3 None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:21.385328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ab799da8-31d1-4834-9812-710191c5ecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:21.478807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 894/1799] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:21 2022] POST /identity/v3/projects => generated 367 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:21.480112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af6ca9-ca9d-46a4-8bf9-bfe61408c9b6 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:21.542959 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 906/1800] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:21.544060 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab799da8-31d1-4834-9812-710191c5ecc3 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:21.559972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab799da8-31d1-4834-9812-710191c5ecc3 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:21.602812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af6ca9-ca9d-46a4-8bf9-bfe61408c9b6 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:21.612238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28be979-d4ca-4770-97ce-ce4b3c5bc998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:21.618479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28be979-d4ca-4770-97ce-ce4b3c5bc998 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.618770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28be979-d4ca-4770-97ce-ce4b3c5bc998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.619048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28be979-d4ca-4770-97ce-ce4b3c5bc998 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.620033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28be979-d4ca-4770-97ce-ce4b3c5bc998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:21.621508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28be979-d4ca-4770-97ce-ce4b3c5bc998 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:21.641459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 895/1801] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:21.646207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28be979-d4ca-4770-97ce-ce4b3c5bc998 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:21.655700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4033cb-9957-4ccb-9060-563285c9108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:21.656397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4033cb-9957-4ccb-9060-563285c9108f admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.657012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4033cb-9957-4ccb-9060-563285c9108f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.657384 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4033cb-9957-4ccb-9060-563285c9108f admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.658367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4033cb-9957-4ccb-9060-563285c9108f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:21.663229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28be979-d4ca-4770-97ce-ce4b3c5bc998 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:21.669904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696b3aa-387b-41a6-bf89-3f007b5d4366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:21.671025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696b3aa-387b-41a6-bf89-3f007b5d4366 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.671348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4033cb-9957-4ccb-9060-563285c9108f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:21.671652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696b3aa-387b-41a6-bf89-3f007b5d4366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.672091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696b3aa-387b-41a6-bf89-3f007b5d4366 admin admin] PATH_INFO: `/v3/projects/ca2e65ee55f142dd9ad4419ab18dcf7d/users/0713ce790d0f46678e6402e290d511b5/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.700492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696b3aa-387b-41a6-bf89-3f007b5d4366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2e65ee55f142dd9ad4419ab18dcf7d, user_id=0713ce790d0f46678e6402e290d511b5, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:21.705063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696b3aa-387b-41a6-bf89-3f007b5d4366 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:21.812958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 896/1802] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:21 2022] PUT /identity/v3/projects/ca2e65ee55f142dd9ad4419ab18dcf7d/users/0713ce790d0f46678e6402e290d511b5/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:21.814711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef445-eca4-4b3f-aa38-62e671a53606 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.815001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef445-eca4-4b3f-aa38-62e671a53606 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.815280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef445-eca4-4b3f-aa38-62e671a53606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.849054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 907/1803] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:21 2022] POST /identity/v3/users => generated 290 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:21.852192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4033cb-9957-4ccb-9060-563285c9108f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:21.854357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9d4033cb-9957-4ccb-9060-563285c9108f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:21.857292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-254ef445-eca4-4b3f-aa38-62e671a53606 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:21.858743 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-254ef445-eca4-4b3f-aa38-62e671a53606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:21.872018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4033cb-9957-4ccb-9060-563285c9108f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:21.944628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 897/1804] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:21.945583 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ef445-eca4-4b3f-aa38-62e671a53606 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:21.950223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:21.950886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.951204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.951449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] PATH_INFO: `/v3/users/67d461a8196745219340ca93abacc849` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.953623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d461a8196745219340ca93abacc849)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:21.956558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:21.958409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ef445-eca4-4b3f-aa38-62e671a53606 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:21.974823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783676e-93b4-4742-a5f0-79a7515d90e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:21.975844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783676e-93b4-4742-a5f0-79a7515d90e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:21.976122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783676e-93b4-4742-a5f0-79a7515d90e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:21.976392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783676e-93b4-4742-a5f0-79a7515d90e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:21.977040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783676e-93b4-4742-a5f0-79a7515d90e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:21.980781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783676e-93b4-4742-a5f0-79a7515d90e8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:22.022563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 898/1805] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:22.022563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783676e-93b4-4742-a5f0-79a7515d90e8 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:22.022563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7783676e-93b4-4742-a5f0-79a7515d90e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:22.101312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783676e-93b4-4742-a5f0-79a7515d90e8 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:22.105382 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2cd72-1372-4baa-99ea-0a49c7f4cf3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:22.106168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2cd72-1372-4baa-99ea-0a49c7f4cf3c service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:22.106447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2cd72-1372-4baa-99ea-0a49c7f4cf3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:22.106716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2cd72-1372-4baa-99ea-0a49c7f4cf3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:22.118388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2cd72-1372-4baa-99ea-0a49c7f4cf3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:22.126037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2cd72-1372-4baa-99ea-0a49c7f4cf3c service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:22.133201 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 899/1806] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:22.134330 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2cd72-1372-4baa-99ea-0a49c7f4cf3c service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:22.153389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2cd72-1372-4baa-99ea-0a49c7f4cf3c service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:22.159500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4c580-f684-40d2-b01a-4336883167e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:22.161291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4c580-f684-40d2-b01a-4336883167e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:22.161589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4c580-f684-40d2-b01a-4336883167e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:22.161901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4c580-f684-40d2-b01a-4336883167e3 admin admin] PATH_INFO: `/v3/projects/bff29d400eb347f9bbfcd2832bffa0a1/users/842a4a9de42a4c7484c0b2344e2ca3d3/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:22.187339 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4c580-f684-40d2-b01a-4336883167e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff29d400eb347f9bbfcd2832bffa0a1, user_id=842a4a9de42a4c7484c0b2344e2ca3d3, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:22.193652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4c580-f684-40d2-b01a-4336883167e3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:22.217408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d461a8196745219340ca93abacc849'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:22.249947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d461a8196745219340ca93abacc849'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:22.315501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 900/1807] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:22 2022] PUT /identity/v3/projects/bff29d400eb347f9bbfcd2832bffa0a1/users/842a4a9de42a4c7484c0b2344e2ca3d3/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:22.316424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4c580-f684-40d2-b01a-4336883167e3 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:22.323092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d461a8196745219340ca93abacc849'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:22.324576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d461a8196745219340ca93abacc849'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:22.332292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bda4c580-f684-40d2-b01a-4336883167e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:22.339535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d461a8196745219340ca93abacc849'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:22.362734 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 908/1808] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:21 2022] DELETE /identity/v3/users/67d461a8196745219340ca93abacc849 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:22.368554 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:22.370900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:22.430543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:22.476749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fdd7c-ea64-4e7b-9860-f0933575e92a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:22.481981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccf073-8c17-4882-8ab5-e5863a717a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:22.482728 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccf073-8c17-4882-8ab5-e5863a717a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:22.483838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccf073-8c17-4882-8ab5-e5863a717a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:22.484221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccf073-8c17-4882-8ab5-e5863a717a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:22.485018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccf073-8c17-4882-8ab5-e5863a717a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:22.486740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccf073-8c17-4882-8ab5-e5863a717a47 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:22.492573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4c580-f684-40d2-b01a-4336883167e3 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:22.498530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 909/1809] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:22.500373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1945abc-f881-4872-b50d-7a1d6b012ef4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:22.500514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1945abc-f881-4872-b50d-7a1d6b012ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:22.501008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1945abc-f881-4872-b50d-7a1d6b012ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:22.522124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b1945abc-f881-4872-b50d-7a1d6b012ef4 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:22.523455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b1945abc-f881-4872-b50d-7a1d6b012ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:22.534495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 910/1810] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:22.537800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1945abc-f881-4872-b50d-7a1d6b012ef4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:22.551438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1945abc-f881-4872-b50d-7a1d6b012ef4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:22.557561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c276de4-f1cc-4dc5-9aba-620a71c1cbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:22.558345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c276de4-f1cc-4dc5-9aba-620a71c1cbab admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:22.558761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c276de4-f1cc-4dc5-9aba-620a71c1cbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:22.559155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c276de4-f1cc-4dc5-9aba-620a71c1cbab admin admin] PATH_INFO: `/v3/projects/bff29d400eb347f9bbfcd2832bffa0a1/users/842a4a9de42a4c7484c0b2344e2ca3d3/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:22.561897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c276de4-f1cc-4dc5-9aba-620a71c1cbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff29d400eb347f9bbfcd2832bffa0a1, user_id=842a4a9de42a4c7484c0b2344e2ca3d3, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:22.566249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c276de4-f1cc-4dc5-9aba-620a71c1cbab admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:22.579935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 911/1811] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:22 2022] PUT /identity/v3/projects/bff29d400eb347f9bbfcd2832bffa0a1/users/842a4a9de42a4c7484c0b2344e2ca3d3/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:22.581165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c276de4-f1cc-4dc5-9aba-620a71c1cbab admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:22.582930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0c276de4-f1cc-4dc5-9aba-620a71c1cbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:22.687747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f80b5-f9dd-4946-bc75-ebe3f163ab1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:22.688412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f80b5-f9dd-4946-bc75-ebe3f163ab1a service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:22.688699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f80b5-f9dd-4946-bc75-ebe3f163ab1a service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:22.689027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f80b5-f9dd-4946-bc75-ebe3f163ab1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:22.690019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:22.691906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bb0f80b5-f9dd-4946-bc75-ebe3f163ab1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:22.710633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f80b5-f9dd-4946-bc75-ebe3f163ab1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:22.714252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f80b5-f9dd-4946-bc75-ebe3f163ab1a service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:22.734644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 901/1812] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:22.736802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e3f31-4761-4482-80b4-dd79aa444056 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:22.737094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e3f31-4761-4482-80b4-dd79aa444056 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:22.737372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e3f31-4761-4482-80b4-dd79aa444056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:22.751338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c276de4-f1cc-4dc5-9aba-620a71c1cbab admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:22.760912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48561f-0608-4ca4-bd77-08f3aef8960a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:22.765685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48561f-0608-4ca4-bd77-08f3aef8960a service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:22.765685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48561f-0608-4ca4-bd77-08f3aef8960a service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:22.765685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48561f-0608-4ca4-bd77-08f3aef8960a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:22.838544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ec0e3f31-4761-4482-80b4-dd79aa444056 None None] MFA Rules not processed for user `842a4a9de42a4c7484c0b2344e2ca3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:22.853095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ec0e3f31-4761-4482-80b4-dd79aa444056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:22.945176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48561f-0608-4ca4-bd77-08f3aef8960a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:22.951330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48561f-0608-4ca4-bd77-08f3aef8960a service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:22.970383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 912/1813] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:23.058890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 902/1814] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:23.071493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48561f-0608-4ca4-bd77-08f3aef8960a service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:23.072956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1c48561f-0608-4ca4-bd77-08f3aef8960a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:23.153805 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48561f-0608-4ca4-bd77-08f3aef8960a service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:23.158214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bcf20-112c-4cc0-8d66-f0b1e9d70d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:23.158831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bcf20-112c-4cc0-8d66-f0b1e9d70d3f service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:23.159115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bcf20-112c-4cc0-8d66-f0b1e9d70d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:23.159392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bcf20-112c-4cc0-8d66-f0b1e9d70d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:23.168598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bcf20-112c-4cc0-8d66-f0b1e9d70d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:23.170053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bcf20-112c-4cc0-8d66-f0b1e9d70d3f service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:23.180409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 913/1815] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:23.369634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e3f31-4761-4482-80b4-dd79aa444056 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:23.372052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ec0e3f31-4761-4482-80b4-dd79aa444056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:23.388528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e3f31-4761-4482-80b4-dd79aa444056 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:23.398521 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92002fcb-25ba-4b64-9031-266734df4218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:23.399484 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92002fcb-25ba-4b64-9031-266734df4218 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:23.401209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92002fcb-25ba-4b64-9031-266734df4218 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:23.401598 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92002fcb-25ba-4b64-9031-266734df4218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:23.407254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-92002fcb-25ba-4b64-9031-266734df4218 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:23.500200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92002fcb-25ba-4b64-9031-266734df4218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:23.501845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92002fcb-25ba-4b64-9031-266734df4218 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:23.516392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 903/1816] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:23.766055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bcf20-112c-4cc0-8d66-f0b1e9d70d3f service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:23.767647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7b0bcf20-112c-4cc0-8d66-f0b1e9d70d3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:23.801676 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bcf20-112c-4cc0-8d66-f0b1e9d70d3f service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:23.812245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:23.813089 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:23.813378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:23.813854 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] PATH_INFO: `/v3/projects/798c8bf087f8499b81057bbf2c532cf7` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:23.815052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=798c8bf087f8499b81057bbf2c532cf7)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:23.818066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:24.047302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798c8bf087f8499b81057bbf2c532cf7', 'request_id': 'req-08dc91c1-8ded-497b-b079-675d0a49d046'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:24.054289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798c8bf087f8499b81057bbf2c532cf7', 'request_id': 'req-08dc91c1-8ded-497b-b079-675d0a49d046'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:24.136844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798c8bf087f8499b81057bbf2c532cf7', 'request_id': 'req-08dc91c1-8ded-497b-b079-675d0a49d046'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:24.149590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] The token cache is being invalidate because project 798c8bf087f8499b81057bbf2c532cf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:24.150004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:24.150738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-08dc91c1-8ded-497b-b079-675d0a49d046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:24.152336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 914/1817] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:23 2022] DELETE /identity/v3/projects/798c8bf087f8499b81057bbf2c532cf7 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:24.156463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92002fcb-25ba-4b64-9031-266734df4218 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:24.167789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92002fcb-25ba-4b64-9031-266734df4218 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:24.257566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:24.258245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:24.258523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:24.258799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] PATH_INFO: `/v3/users/364756511bc64c5c998655055b7cbbb9` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:24.261462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] RBAC: Authorizing `identity:delete_user(user_id=364756511bc64c5c998655055b7cbbb9)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:24.263456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:24.636024 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364756511bc64c5c998655055b7cbbb9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:24.693716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:24.755252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364756511bc64c5c998655055b7cbbb9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:24.755912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364756511bc64c5c998655055b7cbbb9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:24.765877 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364756511bc64c5c998655055b7cbbb9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:24.778863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364756511bc64c5c998655055b7cbbb9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:24.785618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 904/1818] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:24 2022] DELETE /identity/v3/users/364756511bc64c5c998655055b7cbbb9 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:24.814694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b1b7d-172d-4f17-8198-37eaa3982717 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:24.815018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b1b7d-172d-4f17-8198-37eaa3982717 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:24.816553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b1b7d-172d-4f17-8198-37eaa3982717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:24.851213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-485b1b7d-172d-4f17-8198-37eaa3982717 None None] MFA Rules not processed for user `0713ce790d0f46678e6402e290d511b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:24.854857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-485b1b7d-172d-4f17-8198-37eaa3982717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:24.906620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:24.986384 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 915/1819] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:24.996128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:24.999634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:25.071348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e748ed-eb6a-4d37-86cc-3673cbb14deb admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:25.075038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8f3de-d93d-47bf-8142-031ae9bdaf6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:25.075642 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8f3de-d93d-47bf-8142-031ae9bdaf6b service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:25.075930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8f3de-d93d-47bf-8142-031ae9bdaf6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:25.076205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8f3de-d93d-47bf-8142-031ae9bdaf6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:25.086204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8f3de-d93d-47bf-8142-031ae9bdaf6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:25.087977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8f3de-d93d-47bf-8142-031ae9bdaf6b service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:25.099129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 905/1820] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:25.724295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b1b7d-172d-4f17-8198-37eaa3982717 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:25.726433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-485b1b7d-172d-4f17-8198-37eaa3982717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:25.857141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b1b7d-172d-4f17-8198-37eaa3982717 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:25.865746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:25.866431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:25.866873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:25.867153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] PATH_INFO: `/v3/users/76830822b78f4ce7bad5dc87d1b53229` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:25.868296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76830822b78f4ce7bad5dc87d1b53229)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:25.870688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:26.185888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76830822b78f4ce7bad5dc87d1b53229'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:26.206963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76830822b78f4ce7bad5dc87d1b53229'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:26.278119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76830822b78f4ce7bad5dc87d1b53229'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:26.280314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76830822b78f4ce7bad5dc87d1b53229'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:26.316860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76830822b78f4ce7bad5dc87d1b53229'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:26.328941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 916/1821] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:25 2022] DELETE /identity/v3/users/76830822b78f4ce7bad5dc87d1b53229 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:26.330323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d1fc4-9587-4544-84df-ef3f0de4a75f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:26.330817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d1fc4-9587-4544-84df-ef3f0de4a75f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:26.330907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d1fc4-9587-4544-84df-ef3f0de4a75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:26.373489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-557d1fc4-9587-4544-84df-ef3f0de4a75f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:26.376003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-557d1fc4-9587-4544-84df-ef3f0de4a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:26.455746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 906/1822] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:26.494432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:26.496030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:26.555067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d1fc4-9587-4544-84df-ef3f0de4a75f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:26.556920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-557d1fc4-9587-4544-84df-ef3f0de4a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:26.571123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d1fc4-9587-4544-84df-ef3f0de4a75f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:26.584629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:26.587560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:26.591219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:26.591219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] PATH_INFO: `/v3/projects/885e698cb07d4e668697bb2740ac17de` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:26.593165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=885e698cb07d4e668697bb2740ac17de)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:26.595596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:26.600971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f7f6c-673c-4864-92b5-0b366eb88d96 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:26.605648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a068c1-0fa7-43ec-8ebf-62f64a461fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:26.607685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a068c1-0fa7-43ec-8ebf-62f64a461fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:26.608139 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a068c1-0fa7-43ec-8ebf-62f64a461fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:26.608820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a068c1-0fa7-43ec-8ebf-62f64a461fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:26.622503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a068c1-0fa7-43ec-8ebf-62f64a461fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:26.624245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a068c1-0fa7-43ec-8ebf-62f64a461fa8 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:26.648161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 917/1823] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:26.779044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885e698cb07d4e668697bb2740ac17de', 'request_id': 'req-858311ff-1383-4964-bbaa-e1f63ed41463'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:26.893414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885e698cb07d4e668697bb2740ac17de', 'request_id': 'req-858311ff-1383-4964-bbaa-e1f63ed41463'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:26.901506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885e698cb07d4e668697bb2740ac17de', 'request_id': 'req-858311ff-1383-4964-bbaa-e1f63ed41463'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:26.927707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] The token cache is being invalidate because project 885e698cb07d4e668697bb2740ac17de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:26.928305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:26.932260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-858311ff-1383-4964-bbaa-e1f63ed41463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:26.936594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 907/1824] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:26 2022] DELETE /identity/v3/projects/885e698cb07d4e668697bb2740ac17de => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:26.943893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46aa6e-9096-4e82-9ab9-5d393bf2027a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:26.943893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46aa6e-9096-4e82-9ab9-5d393bf2027a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:26.943893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46aa6e-9096-4e82-9ab9-5d393bf2027a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:26.996523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ff46aa6e-9096-4e82-9ab9-5d393bf2027a None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:26.997776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ff46aa6e-9096-4e82-9ab9-5d393bf2027a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:27.063503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 918/1825] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:27.072798 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975da212-4fbe-4f5b-8381-5c412fbab18c None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:27.073285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975da212-4fbe-4f5b-8381-5c412fbab18c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:27.073624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975da212-4fbe-4f5b-8381-5c412fbab18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:27.112005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-975da212-4fbe-4f5b-8381-5c412fbab18c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:27.114959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-975da212-4fbe-4f5b-8381-5c412fbab18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:27.182630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 908/1826] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:27.183966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46aa6e-9096-4e82-9ab9-5d393bf2027a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:27.199170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46aa6e-9096-4e82-9ab9-5d393bf2027a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:27.214792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586060d6-1221-47ba-afa6-e591c6f20fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:27.216196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586060d6-1221-47ba-afa6-e591c6f20fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:27.218027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586060d6-1221-47ba-afa6-e591c6f20fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:27.218518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586060d6-1221-47ba-afa6-e591c6f20fe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:27.219380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586060d6-1221-47ba-afa6-e591c6f20fe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:27.219832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586060d6-1221-47ba-afa6-e591c6f20fe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:27.222077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586060d6-1221-47ba-afa6-e591c6f20fe5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:27.237694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 919/1827] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:49:27.242439 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975da212-4fbe-4f5b-8381-5c412fbab18c None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:27.252841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1274f-0134-4c46-82d9-5451a395f27d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:27.253832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1274f-0134-4c46-82d9-5451a395f27d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:27.253832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1274f-0134-4c46-82d9-5451a395f27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:27.254726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975da212-4fbe-4f5b-8381-5c412fbab18c None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:27.274473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd3dce-903c-46f4-96d5-2f38b8ac1f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:27.275100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd3dce-903c-46f4-96d5-2f38b8ac1f40 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:27.275400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd3dce-903c-46f4-96d5-2f38b8ac1f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:27.275741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd3dce-903c-46f4-96d5-2f38b8ac1f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:27.276288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd3dce-903c-46f4-96d5-2f38b8ac1f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:27.278254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd3dce-903c-46f4-96d5-2f38b8ac1f40 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:27.283941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1dd1274f-0134-4c46-82d9-5451a395f27d None None] MFA Rules not processed for user `0713ce790d0f46678e6402e290d511b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:27.285160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1dd1274f-0134-4c46-82d9-5451a395f27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:27.374734 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 920/1828] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:27.458896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 909/1829] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:27 2022] POST /identity/v3/projects => generated 363 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:27.464592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1274f-0134-4c46-82d9-5451a395f27d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:27.482047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1274f-0134-4c46-82d9-5451a395f27d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:27.574437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abcb7c-6a3d-4281-b8f5-83e6808e9aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:27.576543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abcb7c-6a3d-4281-b8f5-83e6808e9aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:27.576543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abcb7c-6a3d-4281-b8f5-83e6808e9aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:27.577827 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abcb7c-6a3d-4281-b8f5-83e6808e9aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:27.578164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abcb7c-6a3d-4281-b8f5-83e6808e9aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:27.581097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abcb7c-6a3d-4281-b8f5-83e6808e9aa2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:27.654759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd3dce-903c-46f4-96d5-2f38b8ac1f40 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:27.657192 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-46cd3dce-903c-46f4-96d5-2f38b8ac1f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:27.671345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd3dce-903c-46f4-96d5-2f38b8ac1f40 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:27.681871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:27.682742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:27.683177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:27.683591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] PATH_INFO: `/v3/projects/fd843c9df17449e59e3545598686ca05` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:27.684946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd843c9df17449e59e3545598686ca05)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:27.687091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:27.712996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 921/1830] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:27 2022] POST /identity/v3/users => generated 282 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:27.726568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abcb7c-6a3d-4281-b8f5-83e6808e9aa2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:27.737414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abcb7c-6a3d-4281-b8f5-83e6808e9aa2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:27.742239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1401633-1f3b-429c-8a32-9009eb82377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:27.743051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1401633-1f3b-429c-8a32-9009eb82377e admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:27.743398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1401633-1f3b-429c-8a32-9009eb82377e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:27.743740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1401633-1f3b-429c-8a32-9009eb82377e admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:27.744654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1401633-1f3b-429c-8a32-9009eb82377e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:27.746332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1401633-1f3b-429c-8a32-9009eb82377e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:27.759913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 922/1831] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:27.763961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1401633-1f3b-429c-8a32-9009eb82377e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:27.775224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1401633-1f3b-429c-8a32-9009eb82377e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:27.781283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7f6ff-5ebe-4d20-8814-4d57f68e682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:27.782702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7f6ff-5ebe-4d20-8814-4d57f68e682c admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:27.782702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7f6ff-5ebe-4d20-8814-4d57f68e682c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:27.782702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7f6ff-5ebe-4d20-8814-4d57f68e682c admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:27.783516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7f6ff-5ebe-4d20-8814-4d57f68e682c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:27.785804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7f6ff-5ebe-4d20-8814-4d57f68e682c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:27.804556 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 923/1832] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:27.808872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7f6ff-5ebe-4d20-8814-4d57f68e682c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:27.820833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7f6ff-5ebe-4d20-8814-4d57f68e682c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:27.825788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:27.831389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9a7b5-c7a0-48ca-919e-c6e009a46f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:27.832476 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9a7b5-c7a0-48ca-919e-c6e009a46f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:27.832779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9a7b5-c7a0-48ca-919e-c6e009a46f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:27.834761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9a7b5-c7a0-48ca-919e-c6e009a46f11 admin admin] PATH_INFO: `/v3/projects/c7170638677744cea3b2495291d7091c/users/f151d374a3904faea9c7e4963eda0324/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:27.859570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9a7b5-c7a0-48ca-919e-c6e009a46f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7170638677744cea3b2495291d7091c, user_id=f151d374a3904faea9c7e4963eda0324, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:27.866511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9a7b5-c7a0-48ca-919e-c6e009a46f11 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:27.880632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd843c9df17449e59e3545598686ca05', 'request_id': 'req-8dbf9244-482b-4fb4-977c-f8809fe9fa86'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:27.999211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd843c9df17449e59e3545598686ca05', 'request_id': 'req-8dbf9244-482b-4fb4-977c-f8809fe9fa86'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:28.004959 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 924/1833] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:27 2022] PUT /identity/v3/projects/c7170638677744cea3b2495291d7091c/users/f151d374a3904faea9c7e4963eda0324/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:28.012399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd843c9df17449e59e3545598686ca05', 'request_id': 'req-8dbf9244-482b-4fb4-977c-f8809fe9fa86'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:28.016319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687689-b61a-44ac-9984-f0450b038d61 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:28.016808 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687689-b61a-44ac-9984-f0450b038d61 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:28.017223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d687689-b61a-44ac-9984-f0450b038d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:28.026163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] The token cache is being invalidate because project fd843c9df17449e59e3545598686ca05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:28.026519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:28.030798 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:28.033115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 910/1834] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:27 2022] DELETE /identity/v3/projects/fd843c9df17449e59e3545598686ca05 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:28.047203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:28.049480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:28.049649 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4d687689-b61a-44ac-9984-f0450b038d61 None None] MFA Rules not processed for user `f151d374a3904faea9c7e4963eda0324`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:28.050913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4d687689-b61a-44ac-9984-f0450b038d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:28.100948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:28.129367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 925/1835] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:28.130764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf9244-482b-4fb4-977c-f8809fe9fa86 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:28.138497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d687689-b61a-44ac-9984-f0450b038d61 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:28.138497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4d687689-b61a-44ac-9984-f0450b038d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:28.210860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:28.211995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:28.212750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:28.213304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] PATH_INFO: `/v3/users/2ef5b17ce8d34c76bd65bf70665c8451` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:28.214786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef5b17ce8d34c76bd65bf70665c8451)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:28.230455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:28.230644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d687689-b61a-44ac-9984-f0450b038d61 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:28.232093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36b9e7-6d6e-47eb-9cac-8590e2215e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:28.232997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36b9e7-6d6e-47eb-9cac-8590e2215e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:28.233720 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36b9e7-6d6e-47eb-9cac-8590e2215e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:28.233720 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36b9e7-6d6e-47eb-9cac-8590e2215e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:28.234223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36b9e7-6d6e-47eb-9cac-8590e2215e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:28.236173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36b9e7-6d6e-47eb-9cac-8590e2215e21 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:28.353264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 926/1836] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:28 2022] POST /identity/v3/projects => generated 363 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:28.356942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36b9e7-6d6e-47eb-9cac-8590e2215e21 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:28.454925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36b9e7-6d6e-47eb-9cac-8590e2215e21 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:28.456180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef5b17ce8d34c76bd65bf70665c8451'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:28.467552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa819f43-7394-4f7a-afe0-cdb2014211c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:28.468187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa819f43-7394-4f7a-afe0-cdb2014211c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:28.468471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa819f43-7394-4f7a-afe0-cdb2014211c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:28.468747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa819f43-7394-4f7a-afe0-cdb2014211c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:28.469921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa819f43-7394-4f7a-afe0-cdb2014211c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:28.472398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa819f43-7394-4f7a-afe0-cdb2014211c4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:28.578274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef5b17ce8d34c76bd65bf70665c8451'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:28.588707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef5b17ce8d34c76bd65bf70665c8451'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:28.588886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 927/1837] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:28 2022] POST /identity/v3/users => generated 288 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:28.590572 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa819f43-7394-4f7a-afe0-cdb2014211c4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:28.593820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-aa819f43-7394-4f7a-afe0-cdb2014211c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:28.597272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef5b17ce8d34c76bd65bf70665c8451'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:28.620739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa819f43-7394-4f7a-afe0-cdb2014211c4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:28.628125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef5b17ce8d34c76bd65bf70665c8451'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:28.640032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf04e8-b469-4e31-9103-ab044e37be20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:28.640614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 911/1838] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:28 2022] DELETE /identity/v3/users/2ef5b17ce8d34c76bd65bf70665c8451 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:28.640834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf04e8-b469-4e31-9103-ab044e37be20 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:28.641082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf04e8-b469-4e31-9103-ab044e37be20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:28.641353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf04e8-b469-4e31-9103-ab044e37be20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:28.642041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf04e8-b469-4e31-9103-ab044e37be20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:28.643654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf04e8-b469-4e31-9103-ab044e37be20 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:28.668194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 928/1839] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:28.672956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:28.807822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998a393-e1c5-41b5-8240-684d3520759d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:28.818451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a5af8-65c3-499f-9137-710b6420e7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:28.819362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a5af8-65c3-499f-9137-710b6420e7aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:28.819797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a5af8-65c3-499f-9137-710b6420e7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:28.824068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a5af8-65c3-499f-9137-710b6420e7aa admin admin] PATH_INFO: `/v3/projects/54b9050cfaee4fcab5c2ca73e065efe2/users/2658d95dda0f4e0687d3afd49656626b/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:28.839195 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a5af8-65c3-499f-9137-710b6420e7aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b9050cfaee4fcab5c2ca73e065efe2, user_id=2658d95dda0f4e0687d3afd49656626b, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:28.842143 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a5af8-65c3-499f-9137-710b6420e7aa admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:28.989646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 912/1840] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:28 2022] PUT /identity/v3/projects/54b9050cfaee4fcab5c2ca73e065efe2/users/2658d95dda0f4e0687d3afd49656626b/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:28.994798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf04e8-b469-4e31-9103-ab044e37be20 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:29.010584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf04e8-b469-4e31-9103-ab044e37be20 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:29.134484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bbd8f-22e2-4b77-be00-29dac5509334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:29.135438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bbd8f-22e2-4b77-be00-29dac5509334 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:29.135723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bbd8f-22e2-4b77-be00-29dac5509334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:29.136003 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bbd8f-22e2-4b77-be00-29dac5509334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:29.136662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bbd8f-22e2-4b77-be00-29dac5509334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:29.138298 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bbd8f-22e2-4b77-be00-29dac5509334 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:29.153882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 929/1841] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:29.155861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a5af8-65c3-499f-9137-710b6420e7aa admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:29.170768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a5af8-65c3-499f-9137-710b6420e7aa admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:29.189222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b56781-6455-4994-847a-e115f2d6d433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:29.189954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b56781-6455-4994-847a-e115f2d6d433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:29.190247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b56781-6455-4994-847a-e115f2d6d433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:29.190506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b56781-6455-4994-847a-e115f2d6d433 admin admin] PATH_INFO: `/v3/projects/54b9050cfaee4fcab5c2ca73e065efe2/users/2658d95dda0f4e0687d3afd49656626b/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:29.192728 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b56781-6455-4994-847a-e115f2d6d433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b9050cfaee4fcab5c2ca73e065efe2, user_id=2658d95dda0f4e0687d3afd49656626b, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:29.195618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b56781-6455-4994-847a-e115f2d6d433 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:29.208992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 913/1842] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:29 2022] PUT /identity/v3/projects/54b9050cfaee4fcab5c2ca73e065efe2/users/2658d95dda0f4e0687d3afd49656626b/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:29.224293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaeda3d-374a-4685-a77c-1659169dc7ea None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:29.224584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaeda3d-374a-4685-a77c-1659169dc7ea None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:29.225045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaeda3d-374a-4685-a77c-1659169dc7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:29.272785 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9eaeda3d-374a-4685-a77c-1659169dc7ea None None] MFA Rules not processed for user `2658d95dda0f4e0687d3afd49656626b`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:29.272785 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9eaeda3d-374a-4685-a77c-1659169dc7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:29.410341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 930/1843] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:29.428649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b56781-6455-4994-847a-e115f2d6d433 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:29.431032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-86b56781-6455-4994-847a-e115f2d6d433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:29.547876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b56781-6455-4994-847a-e115f2d6d433 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:29.552958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5b83a-7924-4ed2-8607-3943eb19e11f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:29.554033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5b83a-7924-4ed2-8607-3943eb19e11f service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:29.554463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5b83a-7924-4ed2-8607-3943eb19e11f service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:29.555220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5b83a-7924-4ed2-8607-3943eb19e11f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:29.565991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5b83a-7924-4ed2-8607-3943eb19e11f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:29.567696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5b83a-7924-4ed2-8607-3943eb19e11f service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:29.598553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 914/1844] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:29.757815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaeda3d-374a-4685-a77c-1659169dc7ea None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:29.760284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9eaeda3d-374a-4685-a77c-1659169dc7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:29.894296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaeda3d-374a-4685-a77c-1659169dc7ea None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:29.910035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:29.910035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:29.910035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:29.910035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] PATH_INFO: `/v3/projects/d7567d601e194c32bb27e4e0c76f8702` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:29.910378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7567d601e194c32bb27e4e0c76f8702)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:29.910378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:30.032731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5b83a-7924-4ed2-8607-3943eb19e11f service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:30.034673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-34a5b83a-7924-4ed2-8607-3943eb19e11f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:30.054958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5b83a-7924-4ed2-8607-3943eb19e11f service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:30.061605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:30.062669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.063120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.063548 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] PATH_INFO: `/v3/users/0c5c94811fbc4fef824edd6653963c2b` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.065310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5c94811fbc4fef824edd6653963c2b)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:30.068943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:30.115126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7567d601e194c32bb27e4e0c76f8702', 'request_id': 'req-9724e71b-14c0-427e-9ddd-b883399c3b3e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:30.122037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7567d601e194c32bb27e4e0c76f8702', 'request_id': 'req-9724e71b-14c0-427e-9ddd-b883399c3b3e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:30.195798 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:30.224246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7567d601e194c32bb27e4e0c76f8702', 'request_id': 'req-9724e71b-14c0-427e-9ddd-b883399c3b3e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:30.241088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] The token cache is being invalidate because project d7567d601e194c32bb27e4e0c76f8702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:30.241412 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:30.242140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9724e71b-14c0-427e-9ddd-b883399c3b3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:30.244359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 931/1845] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:29 2022] DELETE /identity/v3/projects/d7567d601e194c32bb27e4e0c76f8702 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:30.253569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8b249-8b98-49c5-9378-26dba35d12f8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.257837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8b249-8b98-49c5-9378-26dba35d12f8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.258319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8b249-8b98-49c5-9378-26dba35d12f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.292548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-50d8b249-8b98-49c5-9378-26dba35d12f8 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:30.294561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-50d8b249-8b98-49c5-9378-26dba35d12f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:30.335861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5c94811fbc4fef824edd6653963c2b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:30.358773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 932/1846] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:30.362600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b4822-8e1b-48bb-be9e-ea37e731c1e2 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.362954 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b4822-8e1b-48bb-be9e-ea37e731c1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.363286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b4822-8e1b-48bb-be9e-ea37e731c1e2 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.365140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 933/1847] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:49:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:49:30.366689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfee4f-a8e6-444a-862c-2fa129349c76 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.367049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfee4f-a8e6-444a-862c-2fa129349c76 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.367367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfee4f-a8e6-444a-862c-2fa129349c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.391726 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-03cfee4f-a8e6-444a-862c-2fa129349c76 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:30.392968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-03cfee4f-a8e6-444a-862c-2fa129349c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:30.401390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 934/1848] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:30.405769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfee4f-a8e6-444a-862c-2fa129349c76 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:30.415334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfee4f-a8e6-444a-862c-2fa129349c76 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:30.420126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1e9b0-c6d1-4132-b583-024b5bb461cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:30.421368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1e9b0-c6d1-4132-b583-024b5bb461cd admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.421581 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1e9b0-c6d1-4132-b583-024b5bb461cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.421918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1e9b0-c6d1-4132-b583-024b5bb461cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.422562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1e9b0-c6d1-4132-b583-024b5bb461cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:30.422871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1e9b0-c6d1-4132-b583-024b5bb461cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:30.424492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1e9b0-c6d1-4132-b583-024b5bb461cd admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:30.438537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 935/1849] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:49:30.443069 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1e9b0-c6d1-4132-b583-024b5bb461cd admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:30.453546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1e9b0-c6d1-4132-b583-024b5bb461cd admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:30.458264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787750b-62be-4734-8566-df11edcc682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:30.458873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787750b-62be-4734-8566-df11edcc682b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.459159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787750b-62be-4734-8566-df11edcc682b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.459471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787750b-62be-4734-8566-df11edcc682b admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.460115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787750b-62be-4734-8566-df11edcc682b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:30.462395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787750b-62be-4734-8566-df11edcc682b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:30.477073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5c94811fbc4fef824edd6653963c2b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:30.477824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5c94811fbc4fef824edd6653963c2b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:30.490615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5c94811fbc4fef824edd6653963c2b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:30.505994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef5fbe6f-4b8f-49d1-976d-73ce5bd4975a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5c94811fbc4fef824edd6653963c2b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:30.516287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 915/1850] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:30 2022] DELETE /identity/v3/users/0c5c94811fbc4fef824edd6653963c2b => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:30.516287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa06ec6-c88c-42f5-b4e2-d56f43d7dffc None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.516287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa06ec6-c88c-42f5-b4e2-d56f43d7dffc None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.516287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa06ec6-c88c-42f5-b4e2-d56f43d7dffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.538637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6fa06ec6-c88c-42f5-b4e2-d56f43d7dffc None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:30.539861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6fa06ec6-c88c-42f5-b4e2-d56f43d7dffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:30.630541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 916/1851] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:49:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:30.632529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19502cf8-8af7-42b0-967f-655789fce73b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.632974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19502cf8-8af7-42b0-967f-655789fce73b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.636588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19502cf8-8af7-42b0-967f-655789fce73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.641579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 936/1852] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:30 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:30.646852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787750b-62be-4734-8566-df11edcc682b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:30.648523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e787750b-62be-4734-8566-df11edcc682b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:30.658555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-19502cf8-8af7-42b0-967f-655789fce73b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:30.659759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-19502cf8-8af7-42b0-967f-655789fce73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:30.719755 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787750b-62be-4734-8566-df11edcc682b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:30.720895 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 917/1853] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:30.722880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19502cf8-8af7-42b0-967f-655789fce73b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:30.726693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274d820-13cb-4d05-8d6e-fc4abc070f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:30.727414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274d820-13cb-4d05-8d6e-fc4abc070f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.727798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274d820-13cb-4d05-8d6e-fc4abc070f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.730990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274d820-13cb-4d05-8d6e-fc4abc070f0f admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.730990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274d820-13cb-4d05-8d6e-fc4abc070f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:30.730990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274d820-13cb-4d05-8d6e-fc4abc070f0f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:30.737201 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-19502cf8-8af7-42b0-967f-655789fce73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:30.824171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 937/1854] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:30 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:30.824905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274d820-13cb-4d05-8d6e-fc4abc070f0f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:30.830522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19502cf8-8af7-42b0-967f-655789fce73b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:30.830677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5274d820-13cb-4d05-8d6e-fc4abc070f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:30.834135 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c2859-0c9c-42b9-a87b-67d781d82456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:30.834873 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c2859-0c9c-42b9-a87b-67d781d82456 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.835187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c2859-0c9c-42b9-a87b-67d781d82456 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.835461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c2859-0c9c-42b9-a87b-67d781d82456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.846256 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c2859-0c9c-42b9-a87b-67d781d82456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:30.847985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c2859-0c9c-42b9-a87b-67d781d82456 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:30.862209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 918/1855] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:30.863063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c2859-0c9c-42b9-a87b-67d781d82456 service nova] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:30.878958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c2859-0c9c-42b9-a87b-67d781d82456 service nova] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:30.890450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0f141-1d38-4646-a519-41edaecee606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:30.891288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0f141-1d38-4646-a519-41edaecee606 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.892161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0f141-1d38-4646-a519-41edaecee606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.892715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0f141-1d38-4646-a519-41edaecee606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.893907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0f141-1d38-4646-a519-41edaecee606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:30.896321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0f141-1d38-4646-a519-41edaecee606 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:30.910435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 919/1856] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:30.911378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0f141-1d38-4646-a519-41edaecee606 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:30.917185 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8ae0f141-1d38-4646-a519-41edaecee606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:30.920141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274d820-13cb-4d05-8d6e-fc4abc070f0f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:30.925626 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd0e74-1753-4946-aaf7-c90de3307f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:30.926283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd0e74-1753-4946-aaf7-c90de3307f34 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.926563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd0e74-1753-4946-aaf7-c90de3307f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.926961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd0e74-1753-4946-aaf7-c90de3307f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.937899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd0e74-1753-4946-aaf7-c90de3307f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:30.939345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd0e74-1753-4946-aaf7-c90de3307f34 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:30.943002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0f141-1d38-4646-a519-41edaecee606 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:30.950397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 938/1857] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:30.951301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd0e74-1753-4946-aaf7-c90de3307f34 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:30.959231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:30.959906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.960187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.960495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] PATH_INFO: `/v3/users/f151d374a3904faea9c7e4963eda0324` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.962624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f151d374a3904faea9c7e4963eda0324)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:30.965261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:30.976036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd0e74-1753-4946-aaf7-c90de3307f34 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:30.982697 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16dc6d-444c-4e75-bac9-0c61a825512b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:30.983354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16dc6d-444c-4e75-bac9-0c61a825512b admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:30.983679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16dc6d-444c-4e75-bac9-0c61a825512b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:30.984155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16dc6d-444c-4e75-bac9-0c61a825512b admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:30.986280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16dc6d-444c-4e75-bac9-0c61a825512b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:30.990596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16dc6d-444c-4e75-bac9-0c61a825512b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:31.019216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 939/1858] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:31.024580 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16dc6d-444c-4e75-bac9-0c61a825512b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:31.036199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16dc6d-444c-4e75-bac9-0c61a825512b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:31.041634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9bc3e-d4e4-4b37-a6e9-7e57731ed280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:31.045871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9bc3e-d4e4-4b37-a6e9-7e57731ed280 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:31.045871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9bc3e-d4e4-4b37-a6e9-7e57731ed280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:31.045871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9bc3e-d4e4-4b37-a6e9-7e57731ed280 admin admin] PATH_INFO: `/v3/projects/adbcad03e2fc4a31929d64285b55332c/users/3bb767cb027f4900bc06a5ed7b2e1eb6/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:31.067242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9bc3e-d4e4-4b37-a6e9-7e57731ed280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adbcad03e2fc4a31929d64285b55332c, user_id=3bb767cb027f4900bc06a5ed7b2e1eb6, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:31.071191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9bc3e-d4e4-4b37-a6e9-7e57731ed280 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:31.315347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 940/1859] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:31 2022] PUT /identity/v3/projects/adbcad03e2fc4a31929d64285b55332c/users/3bb767cb027f4900bc06a5ed7b2e1eb6/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:31.328857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3ac9-50a1-479f-9e7f-bff5990382f5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:31.329594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3ac9-50a1-479f-9e7f-bff5990382f5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:31.330141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3ac9-50a1-479f-9e7f-bff5990382f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:31.368300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-48ce3ac9-50a1-479f-9e7f-bff5990382f5 None None] MFA Rules not processed for user `3bb767cb027f4900bc06a5ed7b2e1eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:31.370201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-48ce3ac9-50a1-479f-9e7f-bff5990382f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:31.388839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f151d374a3904faea9c7e4963eda0324'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:31.489015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 941/1860] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:31.490416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3ac9-50a1-479f-9e7f-bff5990382f5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:31.555409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f151d374a3904faea9c7e4963eda0324'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:31.574790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f151d374a3904faea9c7e4963eda0324'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:31.584925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3ac9-50a1-479f-9e7f-bff5990382f5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:31.591118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe95794-22b7-411c-a2f2-4265bf286fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:31.591705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe95794-22b7-411c-a2f2-4265bf286fad admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:31.592036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe95794-22b7-411c-a2f2-4265bf286fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:31.592543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe95794-22b7-411c-a2f2-4265bf286fad admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:31.593015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f151d374a3904faea9c7e4963eda0324'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:31.593543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe95794-22b7-411c-a2f2-4265bf286fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:31.596022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe95794-22b7-411c-a2f2-4265bf286fad admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:31.629719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1e39482e-4e3a-4aa6-afa4-84601b6856a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f151d374a3904faea9c7e4963eda0324'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:31.638958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 920/1861] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:30 2022] DELETE /identity/v3/users/f151d374a3904faea9c7e4963eda0324 => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:31.644065 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b724c-a2aa-43eb-b1ed-35b7f95946f0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:31.646065 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b724c-a2aa-43eb-b1ed-35b7f95946f0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:31.646361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b724c-a2aa-43eb-b1ed-35b7f95946f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:31.668147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d12b724c-a2aa-43eb-b1ed-35b7f95946f0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:31.669383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d12b724c-a2aa-43eb-b1ed-35b7f95946f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:31.716124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 942/1862] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:31 2022] POST /identity/v3/projects => generated 387 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:31.717269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe95794-22b7-411c-a2f2-4265bf286fad admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:31.723013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-afe95794-22b7-411c-a2f2-4265bf286fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:31.755538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 921/1863] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:31.765463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b724c-a2aa-43eb-b1ed-35b7f95946f0 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:31.767209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d12b724c-a2aa-43eb-b1ed-35b7f95946f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:31.813682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe95794-22b7-411c-a2f2-4265bf286fad admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:31.820830 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574564e-8d34-4403-ab99-25f97ae21d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:31.821958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574564e-8d34-4403-ab99-25f97ae21d38 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:31.822409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574564e-8d34-4403-ab99-25f97ae21d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:31.823124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574564e-8d34-4403-ab99-25f97ae21d38 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:31.823835 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574564e-8d34-4403-ab99-25f97ae21d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:31.825644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574564e-8d34-4403-ab99-25f97ae21d38 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:31.856626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b724c-a2aa-43eb-b1ed-35b7f95946f0 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:31.905216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b7eaa-eeaf-4093-99fe-a05cf085549c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:31.905474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b7eaa-eeaf-4093-99fe-a05cf085549c service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:31.905839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b7eaa-eeaf-4093-99fe-a05cf085549c service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:31.906111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b7eaa-eeaf-4093-99fe-a05cf085549c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:31.915718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b7eaa-eeaf-4093-99fe-a05cf085549c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:31.917150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b7eaa-eeaf-4093-99fe-a05cf085549c service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:31.930849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 922/1864] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:31.940633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b7eaa-eeaf-4093-99fe-a05cf085549c service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:31.947163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e77b7eaa-eeaf-4093-99fe-a05cf085549c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:31.962464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 943/1865] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:31 2022] POST /identity/v3/users => generated 294 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:31.966441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574564e-8d34-4403-ab99-25f97ae21d38 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:31.985497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574564e-8d34-4403-ab99-25f97ae21d38 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:31.986656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b7eaa-eeaf-4093-99fe-a05cf085549c service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:31.995128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:31.995996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:31.996338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:31.996714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] PATH_INFO: `/v3/projects/88e93d0a887445a4a5ebefde211dd186` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:31.998079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e93d0a887445a4a5ebefde211dd186)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:31.999502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4ddf9-983d-4b2b-b228-70c89f76e233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:32.000325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4ddf9-983d-4b2b-b228-70c89f76e233 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:32.000610 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4ddf9-983d-4b2b-b228-70c89f76e233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:32.000886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4ddf9-983d-4b2b-b228-70c89f76e233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:32.001546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4ddf9-983d-4b2b-b228-70c89f76e233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:32.003211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4ddf9-983d-4b2b-b228-70c89f76e233 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:32.020475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:32.034028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 944/1866] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:32.038752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4ddf9-983d-4b2b-b228-70c89f76e233 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:32.056901 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4ddf9-983d-4b2b-b228-70c89f76e233 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:32.065582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6504-2313-4922-82b8-bc08636da8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:32.066241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6504-2313-4922-82b8-bc08636da8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:32.066520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6504-2313-4922-82b8-bc08636da8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:32.066792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6504-2313-4922-82b8-bc08636da8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:32.067439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d6504-2313-4922-82b8-bc08636da8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:32.070586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d6504-2313-4922-82b8-bc08636da8b2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:32.090868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 945/1867] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:32.092027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6504-2313-4922-82b8-bc08636da8b2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:32.105228 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:32.118717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6504-2313-4922-82b8-bc08636da8b2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:32.124830 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1c63b-1091-472d-a45b-b07c7f799a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:32.125506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1c63b-1091-472d-a45b-b07c7f799a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:32.125903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1c63b-1091-472d-a45b-b07c7f799a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:32.126175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1c63b-1091-472d-a45b-b07c7f799a19 admin admin] PATH_INFO: `/v3/projects/eb7b75093a2047ac91ae7a92bb5c2a6f/users/e35161d7984d44bbaaa5e0f43482da09/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:32.142933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1c63b-1091-472d-a45b-b07c7f799a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb7b75093a2047ac91ae7a92bb5c2a6f, user_id=e35161d7984d44bbaaa5e0f43482da09, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:32.145660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1c63b-1091-472d-a45b-b07c7f799a19 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:32.353943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 946/1868] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:32 2022] PUT /identity/v3/projects/eb7b75093a2047ac91ae7a92bb5c2a6f/users/e35161d7984d44bbaaa5e0f43482da09/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:32.366496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cc255-1f1f-4dbf-8ecf-c5f1dfff3616 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:32.366786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cc255-1f1f-4dbf-8ecf-c5f1dfff3616 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:32.367246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cc255-1f1f-4dbf-8ecf-c5f1dfff3616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:32.371092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e93d0a887445a4a5ebefde211dd186', 'request_id': 'req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:32.399243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-059cc255-1f1f-4dbf-8ecf-c5f1dfff3616 None None] MFA Rules not processed for user `e35161d7984d44bbaaa5e0f43482da09`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:32.400493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-059cc255-1f1f-4dbf-8ecf-c5f1dfff3616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:32.454638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e93d0a887445a4a5ebefde211dd186', 'request_id': 'req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:32.466007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e93d0a887445a4a5ebefde211dd186', 'request_id': 'req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:32.478046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 947/1869] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:32.479435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cc255-1f1f-4dbf-8ecf-c5f1dfff3616 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:32.484580 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] The token cache is being invalidate because project 88e93d0a887445a4a5ebefde211dd186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:32.486146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:32.487697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:32.490305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 923/1870] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:31 2022] DELETE /identity/v3/projects/88e93d0a887445a4a5ebefde211dd186 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:32.494409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:32.496568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:32.580431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cc255-1f1f-4dbf-8ecf-c5f1dfff3616 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:32.586607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401264c-10fe-45d5-a0e1-fda128ac9021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:32.587810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401264c-10fe-45d5-a0e1-fda128ac9021 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:32.588273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401264c-10fe-45d5-a0e1-fda128ac9021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:32.588760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401264c-10fe-45d5-a0e1-fda128ac9021 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:32.590116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401264c-10fe-45d5-a0e1-fda128ac9021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:32.592286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401264c-10fe-45d5-a0e1-fda128ac9021 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:32.595006 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2419c6-808c-4297-bb1b-0513e6c9cdc1 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:32.600697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:32.601396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:32.601680 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:32.601984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] PATH_INFO: `/v3/users/842a4a9de42a4c7484c0b2344e2ca3d3` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:32.603128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=842a4a9de42a4c7484c0b2344e2ca3d3)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:32.605066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:32.746749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 948/1871] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:32 2022] POST /identity/v3/projects => generated 387 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:32.751043 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401264c-10fe-45d5-a0e1-fda128ac9021 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:32.752777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6401264c-10fe-45d5-a0e1-fda128ac9021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:32.853537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401264c-10fe-45d5-a0e1-fda128ac9021 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:32.859510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5ce63-47ce-4d05-b83e-ec74098d4525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:32.860541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5ce63-47ce-4d05-b83e-ec74098d4525 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:32.861027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5ce63-47ce-4d05-b83e-ec74098d4525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:32.861499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5ce63-47ce-4d05-b83e-ec74098d4525 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:32.862658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5ce63-47ce-4d05-b83e-ec74098d4525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:32.866368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5ce63-47ce-4d05-b83e-ec74098d4525 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:32.897569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842a4a9de42a4c7484c0b2344e2ca3d3'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:33.056522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 949/1872] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:32 2022] POST /identity/v3/users => generated 300 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:33.057356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5ce63-47ce-4d05-b83e-ec74098d4525 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:33.058961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8dd5ce63-47ce-4d05-b83e-ec74098d4525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:33.071273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5ce63-47ce-4d05-b83e-ec74098d4525 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:33.075909 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:33.076516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:33.077442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:33.077804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] PATH_INFO: `/v3/projects/c7170638677744cea3b2495291d7091c` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:33.082137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7170638677744cea3b2495291d7091c)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:33.082137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:33.173177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842a4a9de42a4c7484c0b2344e2ca3d3'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:33.173999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842a4a9de42a4c7484c0b2344e2ca3d3'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:33.186984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842a4a9de42a4c7484c0b2344e2ca3d3'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:33.219595 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842a4a9de42a4c7484c0b2344e2ca3d3'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:33.232257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 924/1873] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:32 2022] DELETE /identity/v3/users/842a4a9de42a4c7484c0b2344e2ca3d3 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:33.233849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:33.235441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:33.239130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7170638677744cea3b2495291d7091c', 'request_id': 'req-236eaa78-f76c-480c-8798-443c501908fa'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:33.246898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7170638677744cea3b2495291d7091c', 'request_id': 'req-236eaa78-f76c-480c-8798-443c501908fa'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:33.322818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd59c9f-c465-446e-b42a-c4d6ea20cc7d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:33.328002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f3ce9-5c22-47bf-a3e8-f2a63c1646dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:33.328653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f3ce9-5c22-47bf-a3e8-f2a63c1646dd admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:33.328935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f3ce9-5c22-47bf-a3e8-f2a63c1646dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:33.329212 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f3ce9-5c22-47bf-a3e8-f2a63c1646dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:33.329925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f3ce9-5c22-47bf-a3e8-f2a63c1646dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:33.331608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f3ce9-5c22-47bf-a3e8-f2a63c1646dd admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:33.347320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 925/1874] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:33.348452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f3ce9-5c22-47bf-a3e8-f2a63c1646dd admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:33.359392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f3ce9-5c22-47bf-a3e8-f2a63c1646dd admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:33.363920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b0098-1db1-4734-a087-746179469df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:33.364569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b0098-1db1-4734-a087-746179469df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:33.364849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b0098-1db1-4734-a087-746179469df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:33.365124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b0098-1db1-4734-a087-746179469df2 admin admin] PATH_INFO: `/v3/projects/2b0c1472465e47daa3d38a2a93ced10f/users/661d2d26882a48c7944150217022c131/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:33.382394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b0098-1db1-4734-a087-746179469df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b0c1472465e47daa3d38a2a93ced10f, user_id=661d2d26882a48c7944150217022c131, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:33.385072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b0098-1db1-4734-a087-746179469df2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:33.433645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7170638677744cea3b2495291d7091c', 'request_id': 'req-236eaa78-f76c-480c-8798-443c501908fa'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:33.459534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] The token cache is being invalidate because project c7170638677744cea3b2495291d7091c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:33.460269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:33.462199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:33.464144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 950/1875] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:33 2022] DELETE /identity/v3/projects/c7170638677744cea3b2495291d7091c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:33.468388 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:33.470159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:33.502163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 926/1876] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:33 2022] PUT /identity/v3/projects/2b0c1472465e47daa3d38a2a93ced10f/users/661d2d26882a48c7944150217022c131/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:33.504732 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b0098-1db1-4734-a087-746179469df2 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:33.506611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-160b0098-1db1-4734-a087-746179469df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:33.619083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b0098-1db1-4734-a087-746179469df2 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:33.631109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236eaa78-f76c-480c-8798-443c501908fa admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:33.633938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ab2a3-9bc0-4680-91cf-fba0fc4b6d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:33.634804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ab2a3-9bc0-4680-91cf-fba0fc4b6d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:33.635289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ab2a3-9bc0-4680-91cf-fba0fc4b6d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:33.635783 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ab2a3-9bc0-4680-91cf-fba0fc4b6d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:33.636637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ab2a3-9bc0-4680-91cf-fba0fc4b6d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:33.639086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ab2a3-9bc0-4680-91cf-fba0fc4b6d20 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:33.650762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:33.652177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:33.652765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:33.658390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] PATH_INFO: `/v3/users/2658d95dda0f4e0687d3afd49656626b` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:33.658390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2658d95dda0f4e0687d3afd49656626b)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:33.658390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:33.665043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 927/1877] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:33.668854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ab2a3-9bc0-4680-91cf-fba0fc4b6d20 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:33.692843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ab2a3-9bc0-4680-91cf-fba0fc4b6d20 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:33.699507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42701e4-65c1-41c4-bf8b-2ccbb8f1b63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:33.700306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42701e4-65c1-41c4-bf8b-2ccbb8f1b63c admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:33.700692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42701e4-65c1-41c4-bf8b-2ccbb8f1b63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:33.701071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42701e4-65c1-41c4-bf8b-2ccbb8f1b63c admin admin] PATH_INFO: `/v3/projects/2b0c1472465e47daa3d38a2a93ced10f/users/661d2d26882a48c7944150217022c131/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:33.704897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42701e4-65c1-41c4-bf8b-2ccbb8f1b63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b0c1472465e47daa3d38a2a93ced10f, user_id=661d2d26882a48c7944150217022c131, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:33.707755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42701e4-65c1-41c4-bf8b-2ccbb8f1b63c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:33.722501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 928/1878] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:33 2022] PUT /identity/v3/projects/2b0c1472465e47daa3d38a2a93ced10f/users/661d2d26882a48c7944150217022c131/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:33.733185 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5ffd-a162-41c3-9a32-721bed58400b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:33.733843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5ffd-a162-41c3-9a32-721bed58400b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:33.734445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5ffd-a162-41c3-9a32-721bed58400b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:33.768189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-8aef5ffd-a162-41c3-9a32-721bed58400b None None] MFA Rules not processed for user `661d2d26882a48c7944150217022c131`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:33.769577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8aef5ffd-a162-41c3-9a32-721bed58400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:34.079515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 929/1879] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:34.079515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef5ffd-a162-41c3-9a32-721bed58400b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:34.079515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8aef5ffd-a162-41c3-9a32-721bed58400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:34.079515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef5ffd-a162-41c3-9a32-721bed58400b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:34.083152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5f1c8-89dd-47b3-9e2a-851b66c3b095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:34.083152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5f1c8-89dd-47b3-9e2a-851b66c3b095 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:34.083152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5f1c8-89dd-47b3-9e2a-851b66c3b095 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:34.083152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5f1c8-89dd-47b3-9e2a-851b66c3b095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:34.085496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5f1c8-89dd-47b3-9e2a-851b66c3b095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:34.085496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5f1c8-89dd-47b3-9e2a-851b66c3b095 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:34.085496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 930/1880] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:34.085496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5f1c8-89dd-47b3-9e2a-851b66c3b095 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:34.085496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-56a5f1c8-89dd-47b3-9e2a-851b66c3b095 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:34.094959 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2658d95dda0f4e0687d3afd49656626b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:34.120618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2658d95dda0f4e0687d3afd49656626b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:34.185522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5f1c8-89dd-47b3-9e2a-851b66c3b095 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:34.231610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:34.232448 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:34.232756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:34.233766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] PATH_INFO: `/v3/projects/bff29d400eb347f9bbfcd2832bffa0a1` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:34.239341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2658d95dda0f4e0687d3afd49656626b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:34.245560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2658d95dda0f4e0687d3afd49656626b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:34.245685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff29d400eb347f9bbfcd2832bffa0a1)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:34.245685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:34.253444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7cf274fc-df5c-4253-b81a-a5bf79d4b76c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2658d95dda0f4e0687d3afd49656626b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:34.265759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 951/1881] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:33 2022] DELETE /identity/v3/users/2658d95dda0f4e0687d3afd49656626b => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:34.425605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:34.455716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff29d400eb347f9bbfcd2832bffa0a1', 'request_id': 'req-6aa6557e-9aa1-4687-9937-cb462ece42ea'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:34.599109 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff29d400eb347f9bbfcd2832bffa0a1', 'request_id': 'req-6aa6557e-9aa1-4687-9937-cb462ece42ea'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:34.606715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff29d400eb347f9bbfcd2832bffa0a1', 'request_id': 'req-6aa6557e-9aa1-4687-9937-cb462ece42ea'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:34.624416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] The token cache is being invalidate because project bff29d400eb347f9bbfcd2832bffa0a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:34.624416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:34.624416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6aa6557e-9aa1-4687-9937-cb462ece42ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:34.624416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 931/1882] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:34 2022] DELETE /identity/v3/projects/bff29d400eb347f9bbfcd2832bffa0a1 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:34.627784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25229563-9107-4335-b326-62c2cce57852 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:34.628283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25229563-9107-4335-b326-62c2cce57852 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:34.628387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25229563-9107-4335-b326-62c2cce57852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:34.657280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-25229563-9107-4335-b326-62c2cce57852 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:34.658520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-25229563-9107-4335-b326-62c2cce57852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:34.723078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 952/1883] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:34.730869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84f526-0bd8-462c-b3cc-7bfb55d6bf39 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:34.731152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84f526-0bd8-462c-b3cc-7bfb55d6bf39 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:34.731422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84f526-0bd8-462c-b3cc-7bfb55d6bf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:34.752589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-fe84f526-0bd8-462c-b3cc-7bfb55d6bf39 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:34.753834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-fe84f526-0bd8-462c-b3cc-7bfb55d6bf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:34.762101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 932/1884] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:34.766935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25229563-9107-4335-b326-62c2cce57852 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:34.776568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25229563-9107-4335-b326-62c2cce57852 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:34.781586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f3b7-f497-4380-a093-0a8431928372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:34.785956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f3b7-f497-4380-a093-0a8431928372 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:34.785956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f3b7-f497-4380-a093-0a8431928372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:34.785956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f3b7-f497-4380-a093-0a8431928372 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:34.785956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81f3b7-f497-4380-a093-0a8431928372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:34.785956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81f3b7-f497-4380-a093-0a8431928372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:34.785956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81f3b7-f497-4380-a093-0a8431928372 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:34.797592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 953/1885] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:49:34.802258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84f526-0bd8-462c-b3cc-7bfb55d6bf39 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:34.812409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84f526-0bd8-462c-b3cc-7bfb55d6bf39 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:34.817107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c8972-5111-4cb2-82f3-599fcf613f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:34.820320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c8972-5111-4cb2-82f3-599fcf613f01 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:34.820320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c8972-5111-4cb2-82f3-599fcf613f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:34.820320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c8972-5111-4cb2-82f3-599fcf613f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:34.820320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c8972-5111-4cb2-82f3-599fcf613f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:34.821011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c8972-5111-4cb2-82f3-599fcf613f01 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:34.854311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f3b7-f497-4380-a093-0a8431928372 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:34.856120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-da81f3b7-f497-4380-a093-0a8431928372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:34.871455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f3b7-f497-4380-a093-0a8431928372 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:34.877335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:34.878116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:34.878405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:34.878681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] PATH_INFO: `/v3/projects/54b9050cfaee4fcab5c2ca73e065efe2` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:34.880840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54b9050cfaee4fcab5c2ca73e065efe2)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:34.882798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:34.952083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 933/1886] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:34 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:34.956298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c8972-5111-4cb2-82f3-599fcf613f01 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.031140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54b9050cfaee4fcab5c2ca73e065efe2', 'request_id': 'req-5ba84d0c-ce64-4704-8867-471b7949eb36'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:35.040395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54b9050cfaee4fcab5c2ca73e065efe2', 'request_id': 'req-5ba84d0c-ce64-4704-8867-471b7949eb36'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:35.041130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c8972-5111-4cb2-82f3-599fcf613f01 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:35.105574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ff336-259f-41e8-9918-6e2d26d1fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:35.106156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ff336-259f-41e8-9918-6e2d26d1fd27 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.106649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ff336-259f-41e8-9918-6e2d26d1fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.106912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ff336-259f-41e8-9918-6e2d26d1fd27 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.107575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ff336-259f-41e8-9918-6e2d26d1fd27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:35.110504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ff336-259f-41e8-9918-6e2d26d1fd27 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:35.186240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54b9050cfaee4fcab5c2ca73e065efe2', 'request_id': 'req-5ba84d0c-ce64-4704-8867-471b7949eb36'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:35.200201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] The token cache is being invalidate because project 54b9050cfaee4fcab5c2ca73e065efe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:35.200549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:35.201237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ba84d0c-ce64-4704-8867-471b7949eb36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:35.204034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 954/1887] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:34 2022] DELETE /identity/v3/projects/54b9050cfaee4fcab5c2ca73e065efe2 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:35.208866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f245ba0-0521-4306-9dbe-4b8643ce798d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.209698 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f245ba0-0521-4306-9dbe-4b8643ce798d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.210077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f245ba0-0521-4306-9dbe-4b8643ce798d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.234833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 934/1888] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:34 2022] POST /identity/v3/users => generated 294 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:35.241564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1f245ba0-0521-4306-9dbe-4b8643ce798d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:35.243057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1f245ba0-0521-4306-9dbe-4b8643ce798d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:35.249668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ff336-259f-41e8-9918-6e2d26d1fd27 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.252707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-240ff336-259f-41e8-9918-6e2d26d1fd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:35.254380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 955/1889] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:35.261693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1a12c-566a-4067-b696-bb70814cff2c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.262012 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1a12c-566a-4067-b696-bb70814cff2c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.262290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1a12c-566a-4067-b696-bb70814cff2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.268124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ff336-259f-41e8-9918-6e2d26d1fd27 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:35.273822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535436cf-2a81-432c-a8ea-b52dc1b72011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:35.274435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535436cf-2a81-432c-a8ea-b52dc1b72011 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.274713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535436cf-2a81-432c-a8ea-b52dc1b72011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.274987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535436cf-2a81-432c-a8ea-b52dc1b72011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.275633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535436cf-2a81-432c-a8ea-b52dc1b72011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:35.277245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535436cf-2a81-432c-a8ea-b52dc1b72011 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:35.288924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 935/1890] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:35.289322 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b7a1a12c-566a-4067-b696-bb70814cff2c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:35.290589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b7a1a12c-566a-4067-b696-bb70814cff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:35.292526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535436cf-2a81-432c-a8ea-b52dc1b72011 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.312165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 956/1891] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:35.312668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1a12c-566a-4067-b696-bb70814cff2c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.314532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535436cf-2a81-432c-a8ea-b52dc1b72011 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:35.328845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5aed3-3b23-47dd-85d6-aee270dea8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:35.329498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5aed3-3b23-47dd-85d6-aee270dea8ec admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.329870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5aed3-3b23-47dd-85d6-aee270dea8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.330213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5aed3-3b23-47dd-85d6-aee270dea8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.332093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5aed3-3b23-47dd-85d6-aee270dea8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:35.337311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5aed3-3b23-47dd-85d6-aee270dea8ec admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:35.341473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1a12c-566a-4067-b696-bb70814cff2c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:35.352803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 936/1892] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:35.356733 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5aed3-3b23-47dd-85d6-aee270dea8ec admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.366830 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e479cc-c50a-4874-b1ff-2277b08c1445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:35.368793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e479cc-c50a-4874-b1ff-2277b08c1445 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.368793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e479cc-c50a-4874-b1ff-2277b08c1445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.368793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e479cc-c50a-4874-b1ff-2277b08c1445 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.368793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e479cc-c50a-4874-b1ff-2277b08c1445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:35.369355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e479cc-c50a-4874-b1ff-2277b08c1445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:35.371405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e479cc-c50a-4874-b1ff-2277b08c1445 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:35.377942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5aed3-3b23-47dd-85d6-aee270dea8ec admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:35.383157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d221f-4b4d-44c1-8c30-f139c3ec102a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:35.384007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d221f-4b4d-44c1-8c30-f139c3ec102a admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.384348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d221f-4b4d-44c1-8c30-f139c3ec102a admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.384624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d221f-4b4d-44c1-8c30-f139c3ec102a admin admin] PATH_INFO: `/v3/projects/38a52dfa5814492cbc47f44005696a36/users/5051de7b6c5941d6b33d6f48f653fe9a/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.386807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 957/1893] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:49:35.394799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e479cc-c50a-4874-b1ff-2277b08c1445 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.408825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e479cc-c50a-4874-b1ff-2277b08c1445 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:35.415803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07eaef7-6299-43a5-ba87-b14e614ce921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:35.416537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d221f-4b4d-44c1-8c30-f139c3ec102a admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a52dfa5814492cbc47f44005696a36, user_id=5051de7b6c5941d6b33d6f48f653fe9a, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:35.416891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07eaef7-6299-43a5-ba87-b14e614ce921 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.417277 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07eaef7-6299-43a5-ba87-b14e614ce921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.417759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07eaef7-6299-43a5-ba87-b14e614ce921 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.418516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07eaef7-6299-43a5-ba87-b14e614ce921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:35.420041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d221f-4b4d-44c1-8c30-f139c3ec102a admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:35.421571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07eaef7-6299-43a5-ba87-b14e614ce921 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:35.565869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 937/1894] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:35 2022] PUT /identity/v3/projects/38a52dfa5814492cbc47f44005696a36/users/5051de7b6c5941d6b33d6f48f653fe9a/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:35.566971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d221f-4b4d-44c1-8c30-f139c3ec102a admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.567871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 958/1895] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:35 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:35.569647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d398fdc-01e0-43bb-adbd-88d36fe66230 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.570014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d398fdc-01e0-43bb-adbd-88d36fe66230 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.570458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d398fdc-01e0-43bb-adbd-88d36fe66230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.572166 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-fb1d221f-4b4d-44c1-8c30-f139c3ec102a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:35.620489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0d398fdc-01e0-43bb-adbd-88d36fe66230 None None] MFA Rules not processed for user `5051de7b6c5941d6b33d6f48f653fe9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:35.622695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0d398fdc-01e0-43bb-adbd-88d36fe66230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:35.683053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d221f-4b4d-44c1-8c30-f139c3ec102a admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:35.736918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49164e9c-3ceb-4278-99bf-48f329aa0b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:35.737282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 959/1896] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:35.737786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49164e9c-3ceb-4278-99bf-48f329aa0b3f service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.738092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49164e9c-3ceb-4278-99bf-48f329aa0b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.738277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49164e9c-3ceb-4278-99bf-48f329aa0b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.740091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-49164e9c-3ceb-4278-99bf-48f329aa0b3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:35.742155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d398fdc-01e0-43bb-adbd-88d36fe66230 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.824615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49164e9c-3ceb-4278-99bf-48f329aa0b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:35.827092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49164e9c-3ceb-4278-99bf-48f329aa0b3f service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:35.828911 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d398fdc-01e0-43bb-adbd-88d36fe66230 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:35.833589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cb47a-78f8-415c-b157-95d283bf1bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:35.834244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cb47a-78f8-415c-b157-95d283bf1bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.834524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cb47a-78f8-415c-b157-95d283bf1bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.834802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cb47a-78f8-415c-b157-95d283bf1bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.835440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cb47a-78f8-415c-b157-95d283bf1bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:35.837509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 938/1897] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:35.837702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cb47a-78f8-415c-b157-95d283bf1bc9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:35.838313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49164e9c-3ceb-4278-99bf-48f329aa0b3f service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.850762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49164e9c-3ceb-4278-99bf-48f329aa0b3f service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:35.855514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57b814-2094-4b0a-b8aa-e18c4b2482e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:35.855960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57b814-2094-4b0a-b8aa-e18c4b2482e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:35.856237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57b814-2094-4b0a-b8aa-e18c4b2482e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:35.856507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57b814-2094-4b0a-b8aa-e18c4b2482e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:35.857133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57b814-2094-4b0a-b8aa-e18c4b2482e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:35.859645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57b814-2094-4b0a-b8aa-e18c4b2482e5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:35.977764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:35.983481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 939/1898] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:35 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:35.987088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 960/1899] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:35 2022] POST /identity/v3/users => generated 283 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:35.987407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57b814-2094-4b0a-b8aa-e18c4b2482e5 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.988177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cb47a-78f8-415c-b157-95d283bf1bc9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:35.999529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cb47a-78f8-415c-b157-95d283bf1bc9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.054305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57b814-2094-4b0a-b8aa-e18c4b2482e5 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.060129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e4ab8-3c37-47ef-8e29-e09c30628869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.060899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e4ab8-3c37-47ef-8e29-e09c30628869 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.061881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e4ab8-3c37-47ef-8e29-e09c30628869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.062168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e4ab8-3c37-47ef-8e29-e09c30628869 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.062803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e4ab8-3c37-47ef-8e29-e09c30628869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:36.064742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e4ab8-3c37-47ef-8e29-e09c30628869 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:36.070035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de853a3-8273-4c56-9665-a099a69d7691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.071051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de853a3-8273-4c56-9665-a099a69d7691 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.071335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de853a3-8273-4c56-9665-a099a69d7691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.071611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de853a3-8273-4c56-9665-a099a69d7691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.072319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de853a3-8273-4c56-9665-a099a69d7691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:36.073950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de853a3-8273-4c56-9665-a099a69d7691 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:36.084912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 961/1900] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:36.089137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de853a3-8273-4c56-9665-a099a69d7691 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:36.099703 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de853a3-8273-4c56-9665-a099a69d7691 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.104458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b21358-f13b-4189-8035-db118fb3b157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.105102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b21358-f13b-4189-8035-db118fb3b157 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.105378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b21358-f13b-4189-8035-db118fb3b157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.105648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b21358-f13b-4189-8035-db118fb3b157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.106310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b21358-f13b-4189-8035-db118fb3b157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:36.107977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b21358-f13b-4189-8035-db118fb3b157 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:36.117998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 962/1901] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:36.121944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b21358-f13b-4189-8035-db118fb3b157 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:36.131787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b21358-f13b-4189-8035-db118fb3b157 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.136609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9bee2-6cc0-4467-ac72-0ff32b9c9dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.137293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9bee2-6cc0-4467-ac72-0ff32b9c9dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.137574 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9bee2-6cc0-4467-ac72-0ff32b9c9dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.137894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9bee2-6cc0-4467-ac72-0ff32b9c9dd4 admin admin] PATH_INFO: `/v3/projects/00eeab850ffb470eaa58033f7e5f95e0/users/96e63c137b4d4c66a983f54a39fa0b01/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.152805 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9bee2-6cc0-4467-ac72-0ff32b9c9dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00eeab850ffb470eaa58033f7e5f95e0, user_id=96e63c137b4d4c66a983f54a39fa0b01, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:36.155565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9bee2-6cc0-4467-ac72-0ff32b9c9dd4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:36.178490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 940/1902] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:35 2022] POST /identity/v3/users => generated 300 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:36.182389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e4ab8-3c37-47ef-8e29-e09c30628869 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:36.192705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e4ab8-3c37-47ef-8e29-e09c30628869 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.197223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a7fa7-16f7-4cc6-8be8-d1faf30fef20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.197923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a7fa7-16f7-4cc6-8be8-d1faf30fef20 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.198204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a7fa7-16f7-4cc6-8be8-d1faf30fef20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.198472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a7fa7-16f7-4cc6-8be8-d1faf30fef20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.199101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a7fa7-16f7-4cc6-8be8-d1faf30fef20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:36.200682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a7fa7-16f7-4cc6-8be8-d1faf30fef20 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:36.212186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 941/1903] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:36.216062 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a7fa7-16f7-4cc6-8be8-d1faf30fef20 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:36.225404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a7fa7-16f7-4cc6-8be8-d1faf30fef20 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.230488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93185b2-0d3e-4e2f-a1a8-f53b62555394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.231105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93185b2-0d3e-4e2f-a1a8-f53b62555394 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.231381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93185b2-0d3e-4e2f-a1a8-f53b62555394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.231656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93185b2-0d3e-4e2f-a1a8-f53b62555394 admin admin] PATH_INFO: `/v3/projects/7eca1dfbea9645cda2c65c9a9ec7f22c/users/db87326fb97d47e6a651b5dbe40fa914/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.247412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93185b2-0d3e-4e2f-a1a8-f53b62555394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eca1dfbea9645cda2c65c9a9ec7f22c, user_id=db87326fb97d47e6a651b5dbe40fa914, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:36.251044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93185b2-0d3e-4e2f-a1a8-f53b62555394 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:36.303483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 963/1904] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:36 2022] PUT /identity/v3/projects/00eeab850ffb470eaa58033f7e5f95e0/users/96e63c137b4d4c66a983f54a39fa0b01/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:36.314834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269e05-574a-4f05-af87-1c6fe7f4e54c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.315121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269e05-574a-4f05-af87-1c6fe7f4e54c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.315402 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269e05-574a-4f05-af87-1c6fe7f4e54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.337208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 942/1905] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:36 2022] PUT /identity/v3/projects/7eca1dfbea9645cda2c65c9a9ec7f22c/users/db87326fb97d47e6a651b5dbe40fa914/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:36.341794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93185b2-0d3e-4e2f-a1a8-f53b62555394 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:36.351146 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-af269e05-574a-4f05-af87-1c6fe7f4e54c None None] MFA Rules not processed for user `96e63c137b4d4c66a983f54a39fa0b01`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:36.352354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-af269e05-574a-4f05-af87-1c6fe7f4e54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:36.421555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93185b2-0d3e-4e2f-a1a8-f53b62555394 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.426501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 964/1906] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:36.430593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269e05-574a-4f05-af87-1c6fe7f4e54c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:36.430761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d50da-02b8-4c8c-9b09-2aa0f048b503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.431440 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d50da-02b8-4c8c-9b09-2aa0f048b503 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.431777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d50da-02b8-4c8c-9b09-2aa0f048b503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.432123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d50da-02b8-4c8c-9b09-2aa0f048b503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.432888 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d50da-02b8-4c8c-9b09-2aa0f048b503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:36.434724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d50da-02b8-4c8c-9b09-2aa0f048b503 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:36.442451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269e05-574a-4f05-af87-1c6fe7f4e54c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.445703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 943/1907] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:36.447274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349411a7-f594-4401-8fd3-bc567370fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.447952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349411a7-f594-4401-8fd3-bc567370fd54 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.448432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349411a7-f594-4401-8fd3-bc567370fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.448608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349411a7-f594-4401-8fd3-bc567370fd54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.449243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349411a7-f594-4401-8fd3-bc567370fd54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:36.449403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d50da-02b8-4c8c-9b09-2aa0f048b503 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:36.451230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349411a7-f594-4401-8fd3-bc567370fd54 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:36.461159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d50da-02b8-4c8c-9b09-2aa0f048b503 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.475407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e37e4-6da4-4cba-9275-e3a93c60e1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.475407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e37e4-6da4-4cba-9275-e3a93c60e1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.475407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e37e4-6da4-4cba-9275-e3a93c60e1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.475407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e37e4-6da4-4cba-9275-e3a93c60e1f8 admin admin] PATH_INFO: `/v3/projects/7eca1dfbea9645cda2c65c9a9ec7f22c/users/db87326fb97d47e6a651b5dbe40fa914/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.476233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e37e4-6da4-4cba-9275-e3a93c60e1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eca1dfbea9645cda2c65c9a9ec7f22c, user_id=db87326fb97d47e6a651b5dbe40fa914, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:36.476233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e37e4-6da4-4cba-9275-e3a93c60e1f8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:36.504622 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 944/1908] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:36 2022] PUT /identity/v3/projects/7eca1dfbea9645cda2c65c9a9ec7f22c/users/db87326fb97d47e6a651b5dbe40fa914/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:36.515926 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b23ac-22c0-4540-9bee-d930ce8cc500 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.516756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b23ac-22c0-4540-9bee-d930ce8cc500 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.517101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b23ac-22c0-4540-9bee-d930ce8cc500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.550762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-0e4b23ac-22c0-4540-9bee-d930ce8cc500 None None] MFA Rules not processed for user `db87326fb97d47e6a651b5dbe40fa914`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:36.552072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0e4b23ac-22c0-4540-9bee-d930ce8cc500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:36.589650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 965/1909] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:36 2022] POST /identity/v3/projects => generated 363 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:36.590863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349411a7-f594-4401-8fd3-bc567370fd54 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:36.679133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 945/1910] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:36.688312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b23ac-22c0-4540-9bee-d930ce8cc500 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:36.752250 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349411a7-f594-4401-8fd3-bc567370fd54 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.757656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee2cca-81f9-4dba-8948-d2e207041130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.758450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee2cca-81f9-4dba-8948-d2e207041130 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.758804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee2cca-81f9-4dba-8948-d2e207041130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.759261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee2cca-81f9-4dba-8948-d2e207041130 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.759975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee2cca-81f9-4dba-8948-d2e207041130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:36.762351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee2cca-81f9-4dba-8948-d2e207041130 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:36.772716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b23ac-22c0-4540-9bee-d930ce8cc500 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.864022 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51718898-c397-4f88-877c-9f44d8ec8a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.865087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51718898-c397-4f88-877c-9f44d8ec8a68 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.867140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51718898-c397-4f88-877c-9f44d8ec8a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.867486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51718898-c397-4f88-877c-9f44d8ec8a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.950126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 966/1911] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:36 2022] POST /identity/v3/users => generated 288 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:36.952499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee2cca-81f9-4dba-8948-d2e207041130 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:36.967797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee2cca-81f9-4dba-8948-d2e207041130 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:36.974991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29953c9-f0c7-4400-adfa-d61478e4238a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:36.983966 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:36.991340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29953c9-f0c7-4400-adfa-d61478e4238a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:36.991712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29953c9-f0c7-4400-adfa-d61478e4238a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:36.992051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29953c9-f0c7-4400-adfa-d61478e4238a admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:36.992843 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29953c9-f0c7-4400-adfa-d61478e4238a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:37.001770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29953c9-f0c7-4400-adfa-d61478e4238a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:37.012450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 967/1912] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:37.012450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29953c9-f0c7-4400-adfa-d61478e4238a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:37.022741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51718898-c397-4f88-877c-9f44d8ec8a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:37.026234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51718898-c397-4f88-877c-9f44d8ec8a68 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:37.033839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29953c9-f0c7-4400-adfa-d61478e4238a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:37.042229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69a4d1-6828-4534-8532-5419c1a1438c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:37.042928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69a4d1-6828-4534-8532-5419c1a1438c admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:37.043211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69a4d1-6828-4534-8532-5419c1a1438c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:37.043498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69a4d1-6828-4534-8532-5419c1a1438c admin admin] PATH_INFO: `/v3/projects/cf5a9563a1d647d781f94379efc9d148/users/d35bd3b489124162a42a64f5ab3a88b3/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:37.050191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 946/1913] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:37.084702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51718898-c397-4f88-877c-9f44d8ec8a68 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:37.087835 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69a4d1-6828-4534-8532-5419c1a1438c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5a9563a1d647d781f94379efc9d148, user_id=d35bd3b489124162a42a64f5ab3a88b3, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:37.091719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69a4d1-6828-4534-8532-5419c1a1438c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:37.100588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51718898-c397-4f88-877c-9f44d8ec8a68 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:37.106067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:37.106719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:37.107011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:37.107385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] PATH_INFO: `/v3/users/5051de7b6c5941d6b33d6f48f653fe9a` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:37.108658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5051de7b6c5941d6b33d6f48f653fe9a)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:37.110641 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:37.224127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 968/1914] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:37 2022] PUT /identity/v3/projects/cf5a9563a1d647d781f94379efc9d148/users/d35bd3b489124162a42a64f5ab3a88b3/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:37.227893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69a4d1-6828-4534-8532-5419c1a1438c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:37.299028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69a4d1-6828-4534-8532-5419c1a1438c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:37.304085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4575b-614c-419a-86f1-825d1e2d22fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:37.304704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4575b-614c-419a-86f1-825d1e2d22fc admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:37.304987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4575b-614c-419a-86f1-825d1e2d22fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:37.305264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4575b-614c-419a-86f1-825d1e2d22fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:37.305948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4575b-614c-419a-86f1-825d1e2d22fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:37.307545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4575b-614c-419a-86f1-825d1e2d22fc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:37.317822 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 969/1915] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:37.321981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4575b-614c-419a-86f1-825d1e2d22fc admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:37.332918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4575b-614c-419a-86f1-825d1e2d22fc admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:37.338101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5db8db-decb-4cfe-b627-480215de28ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:37.338768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5db8db-decb-4cfe-b627-480215de28ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:37.339050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5db8db-decb-4cfe-b627-480215de28ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:37.339329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5db8db-decb-4cfe-b627-480215de28ec admin admin] PATH_INFO: `/v3/projects/cf5a9563a1d647d781f94379efc9d148/users/d35bd3b489124162a42a64f5ab3a88b3/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:37.341329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5db8db-decb-4cfe-b627-480215de28ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5a9563a1d647d781f94379efc9d148, user_id=d35bd3b489124162a42a64f5ab3a88b3, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:37.344808 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5db8db-decb-4cfe-b627-480215de28ec admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:37.355710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 970/1916] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:37 2022] PUT /identity/v3/projects/cf5a9563a1d647d781f94379efc9d148/users/d35bd3b489124162a42a64f5ab3a88b3/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:37.367405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974c8af-d692-4fd6-8cf9-8fe9b2dbec31 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:37.369272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974c8af-d692-4fd6-8cf9-8fe9b2dbec31 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:37.369272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974c8af-d692-4fd6-8cf9-8fe9b2dbec31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:37.397087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f974c8af-d692-4fd6-8cf9-8fe9b2dbec31 None None] MFA Rules not processed for user `d35bd3b489124162a42a64f5ab3a88b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:37.400708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f974c8af-d692-4fd6-8cf9-8fe9b2dbec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:37.422227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5051de7b6c5941d6b33d6f48f653fe9a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:37.529834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 971/1917] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:37.539969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974c8af-d692-4fd6-8cf9-8fe9b2dbec31 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:37.541611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f974c8af-d692-4fd6-8cf9-8fe9b2dbec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:37.555465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5051de7b6c5941d6b33d6f48f653fe9a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:37.556151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5051de7b6c5941d6b33d6f48f653fe9a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:37.565228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5051de7b6c5941d6b33d6f48f653fe9a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:37.581663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-47684e2d-9553-4df1-9f7a-416b44dc19c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5051de7b6c5941d6b33d6f48f653fe9a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:37.609867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 947/1918] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:37 2022] DELETE /identity/v3/users/5051de7b6c5941d6b33d6f48f653fe9a => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:37.622894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974c8af-d692-4fd6-8cf9-8fe9b2dbec31 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:37.627177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2e170-fcb9-4c38-8056-380e3ff5d5fc None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:37.627464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2e170-fcb9-4c38-8056-380e3ff5d5fc None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:37.627745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2e170-fcb9-4c38-8056-380e3ff5d5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:37.669553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4eb2e170-fcb9-4c38-8056-380e3ff5d5fc None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:37.671868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4eb2e170-fcb9-4c38-8056-380e3ff5d5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:37.693494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1606d-5ecc-45a5-86aa-b3f5d1baa913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:37.694302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1606d-5ecc-45a5-86aa-b3f5d1baa913 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:37.694679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1606d-5ecc-45a5-86aa-b3f5d1baa913 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:37.695184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1606d-5ecc-45a5-86aa-b3f5d1baa913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:37.697909 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-61d1606d-5ecc-45a5-86aa-b3f5d1baa913 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:37.790833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 948/1919] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:37.834221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1606d-5ecc-45a5-86aa-b3f5d1baa913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:37.835931 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1606d-5ecc-45a5-86aa-b3f5d1baa913 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:37.865491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2e170-fcb9-4c38-8056-380e3ff5d5fc None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:37.867192 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4eb2e170-fcb9-4c38-8056-380e3ff5d5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:37.891631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 972/1920] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:37.980126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2e170-fcb9-4c38-8056-380e3ff5d5fc None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:37.995677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d623-8e2f-445a-b2b1-20ee3aec49dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:37.996533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d623-8e2f-445a-b2b1-20ee3aec49dc service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:37.996878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d623-8e2f-445a-b2b1-20ee3aec49dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:37.997213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d623-8e2f-445a-b2b1-20ee3aec49dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:38.013713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3d623-8e2f-445a-b2b1-20ee3aec49dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:38.015658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3d623-8e2f-445a-b2b1-20ee3aec49dc service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:38.028126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 949/1921] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:38.232460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1606d-5ecc-45a5-86aa-b3f5d1baa913 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:38.233963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-61d1606d-5ecc-45a5-86aa-b3f5d1baa913 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:38.265852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1606d-5ecc-45a5-86aa-b3f5d1baa913 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:38.272743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692637-c551-4f6a-b82c-77b79d0f4fa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:38.272743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692637-c551-4f6a-b82c-77b79d0f4fa0 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:38.272743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692637-c551-4f6a-b82c-77b79d0f4fa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:38.273929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692637-c551-4f6a-b82c-77b79d0f4fa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:38.275730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dd692637-c551-4f6a-b82c-77b79d0f4fa0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:38.373167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd692637-c551-4f6a-b82c-77b79d0f4fa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:38.377889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd692637-c551-4f6a-b82c-77b79d0f4fa0 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:38.403934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 973/1922] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:38.911694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d623-8e2f-445a-b2b1-20ee3aec49dc service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:38.913326 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bec3d623-8e2f-445a-b2b1-20ee3aec49dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:38.929468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d623-8e2f-445a-b2b1-20ee3aec49dc service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:38.937438 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:38.937438 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:38.937971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:38.938065 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] PATH_INFO: `/v3/projects/38a52dfa5814492cbc47f44005696a36` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:38.939497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38a52dfa5814492cbc47f44005696a36)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:38.942355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:39.103318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38a52dfa5814492cbc47f44005696a36', 'request_id': 'req-15c2c536-a7a2-4a84-a4c1-6288dd537373'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:39.233737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38a52dfa5814492cbc47f44005696a36', 'request_id': 'req-15c2c536-a7a2-4a84-a4c1-6288dd537373'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:39.247421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38a52dfa5814492cbc47f44005696a36', 'request_id': 'req-15c2c536-a7a2-4a84-a4c1-6288dd537373'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:39.256277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] The token cache is being invalidate because project 38a52dfa5814492cbc47f44005696a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:39.256569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:39.257816 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:39.259747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 950/1923] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:38 2022] DELETE /identity/v3/projects/38a52dfa5814492cbc47f44005696a36 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:39.263498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692637-c551-4f6a-b82c-77b79d0f4fa0 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:39.312045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692637-c551-4f6a-b82c-77b79d0f4fa0 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:39.406473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:39.407512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:39.407994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:39.409343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] PATH_INFO: `/v3/users/db87326fb97d47e6a651b5dbe40fa914` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:39.410337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db87326fb97d47e6a651b5dbe40fa914)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:39.415170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:39.759407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db87326fb97d47e6a651b5dbe40fa914'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:39.773685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db87326fb97d47e6a651b5dbe40fa914'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:39.859029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db87326fb97d47e6a651b5dbe40fa914'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:39.860257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db87326fb97d47e6a651b5dbe40fa914'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:39.871022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b42e3020-6618-40fa-9188-8ca0d37f1213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db87326fb97d47e6a651b5dbe40fa914'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:39.889502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 974/1924] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:39 2022] DELETE /identity/v3/users/db87326fb97d47e6a651b5dbe40fa914 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:40.113765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:40.619391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:40.620977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:40.643006 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2c536-a7a2-4a84-a4c1-6288dd537373 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:40.753348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:40.755813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:40.756099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:40.756375 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] PATH_INFO: `/v3/projects/7eca1dfbea9645cda2c65c9a9ec7f22c` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:40.757552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eca1dfbea9645cda2c65c9a9ec7f22c)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:40.759510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:40.930223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eca1dfbea9645cda2c65c9a9ec7f22c', 'request_id': 'req-d55684eb-8323-4bfc-a88f-3bf6e0094141'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:41.005365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eca1dfbea9645cda2c65c9a9ec7f22c', 'request_id': 'req-d55684eb-8323-4bfc-a88f-3bf6e0094141'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:41.015871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eca1dfbea9645cda2c65c9a9ec7f22c', 'request_id': 'req-d55684eb-8323-4bfc-a88f-3bf6e0094141'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:41.052745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] The token cache is being invalidate because project 7eca1dfbea9645cda2c65c9a9ec7f22c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:41.054253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:41.055132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d55684eb-8323-4bfc-a88f-3bf6e0094141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:41.059632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 951/1925] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:40 2022] DELETE /identity/v3/projects/7eca1dfbea9645cda2c65c9a9ec7f22c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:41.066840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac949-4185-40d6-91d3-5dd27ce290fb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:41.067924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac949-4185-40d6-91d3-5dd27ce290fb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:41.068908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac949-4185-40d6-91d3-5dd27ce290fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:41.133518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d88ac949-4185-40d6-91d3-5dd27ce290fb None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:41.135569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d88ac949-4185-40d6-91d3-5dd27ce290fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:41.255784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 975/1926] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:41.262653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdec08-3b0b-45a5-ba59-d32f9100d19c None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:41.263279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdec08-3b0b-45a5-ba59-d32f9100d19c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:41.263627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdec08-3b0b-45a5-ba59-d32f9100d19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:41.320007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-62fdec08-3b0b-45a5-ba59-d32f9100d19c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:41.323415 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-62fdec08-3b0b-45a5-ba59-d32f9100d19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:41.343662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 952/1927] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:41.348115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac949-4185-40d6-91d3-5dd27ce290fb None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:41.383534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac949-4185-40d6-91d3-5dd27ce290fb None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:41.401655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:41.416077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aef292-9256-4cc5-886a-66f1032e1955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:41.420543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aef292-9256-4cc5-886a-66f1032e1955 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:41.421140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aef292-9256-4cc5-886a-66f1032e1955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:41.422425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aef292-9256-4cc5-886a-66f1032e1955 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:41.423314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aef292-9256-4cc5-886a-66f1032e1955 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:41.424527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aef292-9256-4cc5-886a-66f1032e1955 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:41.430040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aef292-9256-4cc5-886a-66f1032e1955 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:41.464260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 976/1928] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:49:41.468098 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdec08-3b0b-45a5-ba59-d32f9100d19c None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:41.481900 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdec08-3b0b-45a5-ba59-d32f9100d19c None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:41.499103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c79a0-943d-43fc-8745-c15d85b2a343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:41.500130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c79a0-943d-43fc-8745-c15d85b2a343 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:41.500407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c79a0-943d-43fc-8745-c15d85b2a343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:41.500677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c79a0-943d-43fc-8745-c15d85b2a343 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:41.501325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c79a0-943d-43fc-8745-c15d85b2a343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:41.504532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c79a0-943d-43fc-8745-c15d85b2a343 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:41.630376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 953/1929] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:41 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:41.634831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aef292-9256-4cc5-886a-66f1032e1955 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:41.648570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aef292-9256-4cc5-886a-66f1032e1955 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:41.710229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c61df6-0936-46c7-995c-e183b36d7990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:41.710850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c61df6-0936-46c7-995c-e183b36d7990 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:41.711141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c61df6-0936-46c7-995c-e183b36d7990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:41.711420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c61df6-0936-46c7-995c-e183b36d7990 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:41.712342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c61df6-0936-46c7-995c-e183b36d7990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:41.714302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c61df6-0936-46c7-995c-e183b36d7990 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:41.814200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 977/1930] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:41 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:41.818372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c79a0-943d-43fc-8745-c15d85b2a343 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:41.833868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c79a0-943d-43fc-8745-c15d85b2a343 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:41.838873 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baab3b7-b33a-47c0-b266-87c3caffc76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:41.839585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baab3b7-b33a-47c0-b266-87c3caffc76f admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:41.846045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baab3b7-b33a-47c0-b266-87c3caffc76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:41.846045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baab3b7-b33a-47c0-b266-87c3caffc76f admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:41.846045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baab3b7-b33a-47c0-b266-87c3caffc76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:41.846045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baab3b7-b33a-47c0-b266-87c3caffc76f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:41.855081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 954/1931] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:41.859634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c61df6-0936-46c7-995c-e183b36d7990 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:41.870418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c61df6-0936-46c7-995c-e183b36d7990 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:41.877110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32fe3a-6279-45d5-9c61-d7d7de7a1df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:41.878039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32fe3a-6279-45d5-9c61-d7d7de7a1df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:41.878328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32fe3a-6279-45d5-9c61-d7d7de7a1df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:41.878742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32fe3a-6279-45d5-9c61-d7d7de7a1df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:41.879420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32fe3a-6279-45d5-9c61-d7d7de7a1df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:41.881197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32fe3a-6279-45d5-9c61-d7d7de7a1df0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:41.899878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 978/1932] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:41.901211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baab3b7-b33a-47c0-b266-87c3caffc76f admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:41.938164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baab3b7-b33a-47c0-b266-87c3caffc76f admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:41.945230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5042eb-b717-4cb0-b3b5-f1162507f21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:41.946139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5042eb-b717-4cb0-b3b5-f1162507f21b admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:41.946570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5042eb-b717-4cb0-b3b5-f1162507f21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:41.947034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5042eb-b717-4cb0-b3b5-f1162507f21b admin admin] PATH_INFO: `/v3/projects/babf2b08a3c74d0e9f1b4896a2162fef/users/46fe815be3e7458a83d1bba91e842400/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:41.966526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5042eb-b717-4cb0-b3b5-f1162507f21b admin admin] RBAC: Authorizing `identity:create_grant(project_id=babf2b08a3c74d0e9f1b4896a2162fef, user_id=46fe815be3e7458a83d1bba91e842400, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:41.969364 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5042eb-b717-4cb0-b3b5-f1162507f21b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:42.180913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc2b71-68ce-4829-b2d1-0b3e18bf55ec None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:42.181430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc2b71-68ce-4829-b2d1-0b3e18bf55ec None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:42.181592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc2b71-68ce-4829-b2d1-0b3e18bf55ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:42.181807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 955/1933] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:41 2022] PUT /identity/v3/projects/babf2b08a3c74d0e9f1b4896a2162fef/users/46fe815be3e7458a83d1bba91e842400/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:42.218185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-bcfc2b71-68ce-4829-b2d1-0b3e18bf55ec None None] MFA Rules not processed for user `46fe815be3e7458a83d1bba91e842400`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:42.220475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bcfc2b71-68ce-4829-b2d1-0b3e18bf55ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:42.348503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 979/1934] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:42.352902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5042eb-b717-4cb0-b3b5-f1162507f21b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:42.438197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5042eb-b717-4cb0-b3b5-f1162507f21b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:42.457946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5362e2-9957-420c-9cf5-712ec3fb9eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:42.457946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5362e2-9957-420c-9cf5-712ec3fb9eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:42.457946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5362e2-9957-420c-9cf5-712ec3fb9eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:42.457946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5362e2-9957-420c-9cf5-712ec3fb9eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:42.463782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5362e2-9957-420c-9cf5-712ec3fb9eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:42.463782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5362e2-9957-420c-9cf5-712ec3fb9eb9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:42.616372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 956/1935] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:42 2022] POST /identity/v3/projects => generated 385 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:42.620387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc2b71-68ce-4829-b2d1-0b3e18bf55ec None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:42.634131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc2b71-68ce-4829-b2d1-0b3e18bf55ec None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:42.722002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f3961-ee8f-40e8-a3b5-e31bf8fe271a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:42.722698 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f3961-ee8f-40e8-a3b5-e31bf8fe271a admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:42.723072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f3961-ee8f-40e8-a3b5-e31bf8fe271a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:42.723632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f3961-ee8f-40e8-a3b5-e31bf8fe271a admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:42.724337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f3961-ee8f-40e8-a3b5-e31bf8fe271a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:42.727936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f3961-ee8f-40e8-a3b5-e31bf8fe271a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:42.900504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5362e2-9957-420c-9cf5-712ec3fb9eb9 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:42.910915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 980/1936] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:42 2022] POST /identity/v3/users => generated 299 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:42.913083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5362e2-9957-420c-9cf5-712ec3fb9eb9 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:42.927062 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf37b7-58a3-49f9-8e62-f507d198e60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:42.927738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf37b7-58a3-49f9-8e62-f507d198e60e admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:42.928091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf37b7-58a3-49f9-8e62-f507d198e60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:42.928479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf37b7-58a3-49f9-8e62-f507d198e60e admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:42.929243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf37b7-58a3-49f9-8e62-f507d198e60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:42.931238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf37b7-58a3-49f9-8e62-f507d198e60e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:42.943934 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 957/1937] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:42.948192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f3961-ee8f-40e8-a3b5-e31bf8fe271a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:42.976132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f3961-ee8f-40e8-a3b5-e31bf8fe271a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:42.976132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73679e42-58c4-4a08-8c93-3a3e262667cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:42.976132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679e42-58c4-4a08-8c93-3a3e262667cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:42.976132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679e42-58c4-4a08-8c93-3a3e262667cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:42.976475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679e42-58c4-4a08-8c93-3a3e262667cb admin admin] PATH_INFO: `/v3/projects/9366aecb39424a21aa5a2b8891f99e7c/users/5bb282c105ab4bc39a7329fc44191b85/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:42.990219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf37b7-58a3-49f9-8e62-f507d198e60e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:42.992806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a9cf37b7-58a3-49f9-8e62-f507d198e60e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:42.993181 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73679e42-58c4-4a08-8c93-3a3e262667cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9366aecb39424a21aa5a2b8891f99e7c, user_id=5bb282c105ab4bc39a7329fc44191b85, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:42.996134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73679e42-58c4-4a08-8c93-3a3e262667cb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:43.069543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf37b7-58a3-49f9-8e62-f507d198e60e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:43.073501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6acf4-8cdb-4ead-bcf9-c72d50df5fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:43.074164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6acf4-8cdb-4ead-bcf9-c72d50df5fbd service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:43.074443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6acf4-8cdb-4ead-bcf9-c72d50df5fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:43.074714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6acf4-8cdb-4ead-bcf9-c72d50df5fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:43.077276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-baf6acf4-8cdb-4ead-bcf9-c72d50df5fbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:43.134235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 981/1938] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:42 2022] PUT /identity/v3/projects/9366aecb39424a21aa5a2b8891f99e7c/users/5bb282c105ab4bc39a7329fc44191b85/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:43.138307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73679e42-58c4-4a08-8c93-3a3e262667cb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:43.166577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6acf4-8cdb-4ead-bcf9-c72d50df5fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:43.168310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6acf4-8cdb-4ead-bcf9-c72d50df5fbd service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:43.186668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 958/1939] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:43.250169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73679e42-58c4-4a08-8c93-3a3e262667cb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:43.256615 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b4466-341a-4590-96e2-73bdb5fc44dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:43.258383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b4466-341a-4590-96e2-73bdb5fc44dd admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:43.262334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b4466-341a-4590-96e2-73bdb5fc44dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:43.262334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b4466-341a-4590-96e2-73bdb5fc44dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:43.262334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b4466-341a-4590-96e2-73bdb5fc44dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:43.262334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b4466-341a-4590-96e2-73bdb5fc44dd admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:43.277512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 982/1940] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:43.283922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6acf4-8cdb-4ead-bcf9-c72d50df5fbd service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:43.301518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6acf4-8cdb-4ead-bcf9-c72d50df5fbd service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:43.309579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64854a67-97c9-4a94-bf80-d2db6c3c2beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:43.310513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64854a67-97c9-4a94-bf80-d2db6c3c2beb admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:43.311228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64854a67-97c9-4a94-bf80-d2db6c3c2beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:43.311505 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64854a67-97c9-4a94-bf80-d2db6c3c2beb admin admin] PATH_INFO: `/v3/projects/9366aecb39424a21aa5a2b8891f99e7c/users/5bb282c105ab4bc39a7329fc44191b85/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:43.314931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64854a67-97c9-4a94-bf80-d2db6c3c2beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9366aecb39424a21aa5a2b8891f99e7c, user_id=5bb282c105ab4bc39a7329fc44191b85, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:43.322385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64854a67-97c9-4a94-bf80-d2db6c3c2beb admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:43.347981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 959/1941] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:43 2022] PUT /identity/v3/projects/9366aecb39424a21aa5a2b8891f99e7c/users/5bb282c105ab4bc39a7329fc44191b85/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:43.349503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c5a08-e057-4aeb-9693-b570eda81dbd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:43.349846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c5a08-e057-4aeb-9693-b570eda81dbd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:43.350129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c5a08-e057-4aeb-9693-b570eda81dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:43.391342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b24c5a08-e057-4aeb-9693-b570eda81dbd None None] MFA Rules not processed for user `5bb282c105ab4bc39a7329fc44191b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:43.393950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b24c5a08-e057-4aeb-9693-b570eda81dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:43.481746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 983/1942] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:43.767875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64854a67-97c9-4a94-bf80-d2db6c3c2beb admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:43.771817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-64854a67-97c9-4a94-bf80-d2db6c3c2beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:43.868054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64854a67-97c9-4a94-bf80-d2db6c3c2beb admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:43.871758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be694fa-7a6e-4e59-bab8-50eb3a5738ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:43.872368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be694fa-7a6e-4e59-bab8-50eb3a5738ee service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:43.872655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be694fa-7a6e-4e59-bab8-50eb3a5738ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:43.872941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be694fa-7a6e-4e59-bab8-50eb3a5738ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:43.888390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be694fa-7a6e-4e59-bab8-50eb3a5738ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:43.890000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be694fa-7a6e-4e59-bab8-50eb3a5738ee service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:43.900804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 960/1943] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:44.435137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:45.178407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:45.351146 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c5a08-e057-4aeb-9693-b570eda81dbd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:45.421148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c5a08-e057-4aeb-9693-b570eda81dbd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:45.426020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:45.426824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:45.427172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:45.427514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] PATH_INFO: `/v3/users/46fe815be3e7458a83d1bba91e842400` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:45.428702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46fe815be3e7458a83d1bba91e842400)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:45.430770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:45.620595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46fe815be3e7458a83d1bba91e842400'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:45.632932 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46fe815be3e7458a83d1bba91e842400'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:45.703455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46fe815be3e7458a83d1bba91e842400'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:45.704092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46fe815be3e7458a83d1bba91e842400'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:45.711360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46fe815be3e7458a83d1bba91e842400'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:45.718215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 984/1944] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:45 2022] DELETE /identity/v3/users/46fe815be3e7458a83d1bba91e842400 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:45.722752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa05a4-fc76-4730-9d9a-ca66463ed7fb None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:45.723031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa05a4-fc76-4730-9d9a-ca66463ed7fb None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:45.723303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa05a4-fc76-4730-9d9a-ca66463ed7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:45.743731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-42fa05a4-fc76-4730-9d9a-ca66463ed7fb None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:45.745085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-42fa05a4-fc76-4730-9d9a-ca66463ed7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:45.804849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 961/1945] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:45.814878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:45.816375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:45.877866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727150d-30aa-4e48-b88d-d748b0b53895 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:45.881462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8484154-04cc-4681-917e-eaf9c3efc346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:45.882102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8484154-04cc-4681-917e-eaf9c3efc346 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:45.882390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8484154-04cc-4681-917e-eaf9c3efc346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:45.882667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8484154-04cc-4681-917e-eaf9c3efc346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:45.892978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8484154-04cc-4681-917e-eaf9c3efc346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:45.894487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8484154-04cc-4681-917e-eaf9c3efc346 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:45.904063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 985/1946] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:46.450515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa05a4-fc76-4730-9d9a-ca66463ed7fb None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:46.450515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-42fa05a4-fc76-4730-9d9a-ca66463ed7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:46.464856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa05a4-fc76-4730-9d9a-ca66463ed7fb None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:46.471388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:46.476040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:46.476040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:46.476040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] PATH_INFO: `/v3/projects/babf2b08a3c74d0e9f1b4896a2162fef` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:46.476040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=babf2b08a3c74d0e9f1b4896a2162fef)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:46.476964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:46.617378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'babf2b08a3c74d0e9f1b4896a2162fef', 'request_id': 'req-7c5733e0-a862-40b2-840e-9882ff24df88'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:46.735633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'babf2b08a3c74d0e9f1b4896a2162fef', 'request_id': 'req-7c5733e0-a862-40b2-840e-9882ff24df88'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:46.742252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'babf2b08a3c74d0e9f1b4896a2162fef', 'request_id': 'req-7c5733e0-a862-40b2-840e-9882ff24df88'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:46.754808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] The token cache is being invalidate because project babf2b08a3c74d0e9f1b4896a2162fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:46.755097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:46.755759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:46.757263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 962/1947] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:46 2022] DELETE /identity/v3/projects/babf2b08a3c74d0e9f1b4896a2162fef => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:46.764293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8484154-04cc-4681-917e-eaf9c3efc346 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:46.774737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8484154-04cc-4681-917e-eaf9c3efc346 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:46.832474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:46.833107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:46.833409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:46.833672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] PATH_INFO: `/v3/users/5bb282c105ab4bc39a7329fc44191b85` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:46.834812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bb282c105ab4bc39a7329fc44191b85)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:46.836741 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:47.031711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bb282c105ab4bc39a7329fc44191b85'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:47.044620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bb282c105ab4bc39a7329fc44191b85'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:47.104020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:47.132300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bb282c105ab4bc39a7329fc44191b85'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:47.133020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bb282c105ab4bc39a7329fc44191b85'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:47.140931 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b5d40a83-6ab2-4047-a6e4-1711cfe24a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bb282c105ab4bc39a7329fc44191b85'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:47.150603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 986/1948] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:46 2022] DELETE /identity/v3/users/5bb282c105ab4bc39a7329fc44191b85 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:47.216366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:47.713959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:47.713959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:47.772128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5733e0-a862-40b2-840e-9882ff24df88 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:47.831958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:47.832695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:47.833276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:47.833964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] PATH_INFO: `/v3/projects/9366aecb39424a21aa5a2b8891f99e7c` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:47.836207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9366aecb39424a21aa5a2b8891f99e7c)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:47.838505 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:47.995498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9366aecb39424a21aa5a2b8891f99e7c', 'request_id': 'req-8859a907-e5d8-47ed-8380-466cf4686b8f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:48.178726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9366aecb39424a21aa5a2b8891f99e7c', 'request_id': 'req-8859a907-e5d8-47ed-8380-466cf4686b8f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:48.185533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9366aecb39424a21aa5a2b8891f99e7c', 'request_id': 'req-8859a907-e5d8-47ed-8380-466cf4686b8f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:48.197943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] The token cache is being invalidate because project 9366aecb39424a21aa5a2b8891f99e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:48.198233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:48.199180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8859a907-e5d8-47ed-8380-466cf4686b8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:48.200705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 963/1949] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:47 2022] DELETE /identity/v3/projects/9366aecb39424a21aa5a2b8891f99e7c => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:48.205924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1871d05-37f2-49e6-a8a8-874021110a06 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:48.206397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1871d05-37f2-49e6-a8a8-874021110a06 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:48.206511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1871d05-37f2-49e6-a8a8-874021110a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:48.226839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e1871d05-37f2-49e6-a8a8-874021110a06 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:48.228083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e1871d05-37f2-49e6-a8a8-874021110a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:48.295964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 987/1950] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:48.298340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b64096-367f-4f61-acfd-bb39157ddd6e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:48.298667 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b64096-367f-4f61-acfd-bb39157ddd6e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:48.298943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b64096-367f-4f61-acfd-bb39157ddd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:48.318917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e0b64096-367f-4f61-acfd-bb39157ddd6e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:48.320100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e0b64096-367f-4f61-acfd-bb39157ddd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:48.328026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 964/1951] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:48.332551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1871d05-37f2-49e6-a8a8-874021110a06 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:48.342567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1871d05-37f2-49e6-a8a8-874021110a06 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:48.353603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f563422-0b1b-40cd-85b0-5205e48eabab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:48.353603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f563422-0b1b-40cd-85b0-5205e48eabab admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:48.353603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f563422-0b1b-40cd-85b0-5205e48eabab admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:48.353603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f563422-0b1b-40cd-85b0-5205e48eabab admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:48.354075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f563422-0b1b-40cd-85b0-5205e48eabab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:48.354075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f563422-0b1b-40cd-85b0-5205e48eabab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:48.355602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f563422-0b1b-40cd-85b0-5205e48eabab admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:48.372066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 988/1952] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:49:48.373114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b64096-367f-4f61-acfd-bb39157ddd6e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:48.383505 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b64096-367f-4f61-acfd-bb39157ddd6e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:48.388201 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80626aa-858f-4c4a-ba6a-bfa079c54c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:48.388804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80626aa-858f-4c4a-ba6a-bfa079c54c2c admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:48.389078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80626aa-858f-4c4a-ba6a-bfa079c54c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:48.389347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80626aa-858f-4c4a-ba6a-bfa079c54c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:48.390001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80626aa-858f-4c4a-ba6a-bfa079c54c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:48.392052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80626aa-858f-4c4a-ba6a-bfa079c54c2c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:48.567956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 965/1953] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:48 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:48.569357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f563422-0b1b-40cd-85b0-5205e48eabab admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:48.580160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f563422-0b1b-40cd-85b0-5205e48eabab admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:48.663966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fd919-db99-4092-8760-01f017d341ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:48.664579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fd919-db99-4092-8760-01f017d341ea admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:48.664904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fd919-db99-4092-8760-01f017d341ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:48.665181 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fd919-db99-4092-8760-01f017d341ea admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:48.666093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fd919-db99-4092-8760-01f017d341ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:48.667860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fd919-db99-4092-8760-01f017d341ea admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:48.782550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 989/1954] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:48 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:48.786782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80626aa-858f-4c4a-ba6a-bfa079c54c2c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:48.797321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80626aa-858f-4c4a-ba6a-bfa079c54c2c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:48.811047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14c8c-c21e-4d27-9844-5416f4e65374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:48.811781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14c8c-c21e-4d27-9844-5416f4e65374 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:48.812754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14c8c-c21e-4d27-9844-5416f4e65374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:48.813251 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14c8c-c21e-4d27-9844-5416f4e65374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:48.814358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d14c8c-c21e-4d27-9844-5416f4e65374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:48.816700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d14c8c-c21e-4d27-9844-5416f4e65374 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:48.828533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 966/1955] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:48.832452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fd919-db99-4092-8760-01f017d341ea admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:48.843880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fd919-db99-4092-8760-01f017d341ea admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:48.848777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac5693-9c96-4e27-8763-28bf2accb515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:48.849399 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac5693-9c96-4e27-8763-28bf2accb515 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:48.849671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac5693-9c96-4e27-8763-28bf2accb515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:48.849976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac5693-9c96-4e27-8763-28bf2accb515 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:48.850741 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac5693-9c96-4e27-8763-28bf2accb515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:48.852687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac5693-9c96-4e27-8763-28bf2accb515 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:48.864517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 990/1956] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:48.872697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14c8c-c21e-4d27-9844-5416f4e65374 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:48.883958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14c8c-c21e-4d27-9844-5416f4e65374 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:48.888644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f487b-5b46-43c4-8feb-02dca2c976f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:48.889465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f487b-5b46-43c4-8feb-02dca2c976f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:48.889761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f487b-5b46-43c4-8feb-02dca2c976f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:48.890934 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f487b-5b46-43c4-8feb-02dca2c976f3 admin admin] PATH_INFO: `/v3/projects/051134557f784863b566fb376a71ef49/users/81a034ed67914a038386462cc6424314/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:48.892122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:48.907682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f487b-5b46-43c4-8feb-02dca2c976f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=051134557f784863b566fb376a71ef49, user_id=81a034ed67914a038386462cc6424314, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:48.910462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f487b-5b46-43c4-8feb-02dca2c976f3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:49.075251 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 967/1957] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:48 2022] PUT /identity/v3/projects/051134557f784863b566fb376a71ef49/users/81a034ed67914a038386462cc6424314/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:49.080737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9dfb3-3ec1-495a-81b8-a7b074117034 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:49.081131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9dfb3-3ec1-495a-81b8-a7b074117034 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:49.081421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9dfb3-3ec1-495a-81b8-a7b074117034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:49.100813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e8a9dfb3-3ec1-495a-81b8-a7b074117034 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:49.102175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e8a9dfb3-3ec1-495a-81b8-a7b074117034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:49.170371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 991/1958] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:49.181184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f487b-5b46-43c4-8feb-02dca2c976f3 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:49.183129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-fe2f487b-5b46-43c4-8feb-02dca2c976f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:49.243985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f487b-5b46-43c4-8feb-02dca2c976f3 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:49.247798 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167d908-f346-4da6-844e-229786fe725a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:49.248425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167d908-f346-4da6-844e-229786fe725a service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:49.248698 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167d908-f346-4da6-844e-229786fe725a service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:49.248963 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167d908-f346-4da6-844e-229786fe725a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:49.259752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167d908-f346-4da6-844e-229786fe725a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:49.261520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167d908-f346-4da6-844e-229786fe725a service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:49.271374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 968/1959] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:49.480296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9dfb3-3ec1-495a-81b8-a7b074117034 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:49.482872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e8a9dfb3-3ec1-495a-81b8-a7b074117034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:49.505271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9dfb3-3ec1-495a-81b8-a7b074117034 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:49.512775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:49.513426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:49.513711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:49.514034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] PATH_INFO: `/v3/users/3bb767cb027f4900bc06a5ed7b2e1eb6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:49.515227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bb767cb027f4900bc06a5ed7b2e1eb6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:49.517961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:49.692316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bb767cb027f4900bc06a5ed7b2e1eb6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:49.711741 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bb767cb027f4900bc06a5ed7b2e1eb6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:49.727585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:49.772780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bb767cb027f4900bc06a5ed7b2e1eb6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:49.773696 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bb767cb027f4900bc06a5ed7b2e1eb6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:49.781229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bb767cb027f4900bc06a5ed7b2e1eb6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:49.787258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 992/1960] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:49 2022] DELETE /identity/v3/users/3bb767cb027f4900bc06a5ed7b2e1eb6 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:49.791722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdcb7b-e6d1-4518-843f-fe8c83e7c2ff None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:49.792185 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdcb7b-e6d1-4518-843f-fe8c83e7c2ff None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:49.792276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdcb7b-e6d1-4518-843f-fe8c83e7c2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:49.813501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-7ccdcb7b-e6d1-4518-843f-fe8c83e7c2ff None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:49.814797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7ccdcb7b-e6d1-4518-843f-fe8c83e7c2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:49.911619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 969/1961] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:49.928086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:49.929925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:50.008894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d09187-e6a9-4e9c-85a1-1694b3e675e4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:50.013771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c840e-7ddb-4e37-b896-cb71a3f497eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:50.014560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c840e-7ddb-4e37-b896-cb71a3f497eb service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:50.018766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c840e-7ddb-4e37-b896-cb71a3f497eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:50.018766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c840e-7ddb-4e37-b896-cb71a3f497eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:50.025368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c840e-7ddb-4e37-b896-cb71a3f497eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:50.027380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c840e-7ddb-4e37-b896-cb71a3f497eb service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:50.045755 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 993/1962] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:50.965148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:50.966052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdcb7b-e6d1-4518-843f-fe8c83e7c2ff None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:50.967486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7ccdcb7b-e6d1-4518-843f-fe8c83e7c2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:50.980464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdcb7b-e6d1-4518-843f-fe8c83e7c2ff None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:50.986701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:50.987554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:50.987660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:50.987901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] PATH_INFO: `/v3/projects/adbcad03e2fc4a31929d64285b55332c` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:50.989114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=adbcad03e2fc4a31929d64285b55332c)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:50.991132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:51.146388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adbcad03e2fc4a31929d64285b55332c', 'request_id': 'req-df3649f8-2a63-4dc0-9dde-3b998b156a8c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:51.172465 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bba7e-d1ce-462b-b517-9c3e5054ce7d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:51.172977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bba7e-d1ce-462b-b517-9c3e5054ce7d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:51.173319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bba7e-d1ce-462b-b517-9c3e5054ce7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:51.208176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6d4bba7e-d1ce-462b-b517-9c3e5054ce7d None None] MFA Rules not processed for user `81a034ed67914a038386462cc6424314`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:51.209792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6d4bba7e-d1ce-462b-b517-9c3e5054ce7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:51.253488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adbcad03e2fc4a31929d64285b55332c', 'request_id': 'req-df3649f8-2a63-4dc0-9dde-3b998b156a8c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:51.263296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adbcad03e2fc4a31929d64285b55332c', 'request_id': 'req-df3649f8-2a63-4dc0-9dde-3b998b156a8c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:51.281702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] The token cache is being invalidate because project adbcad03e2fc4a31929d64285b55332c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:51.282093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:51.283050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:51.287759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 970/1963] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:50 2022] DELETE /identity/v3/projects/adbcad03e2fc4a31929d64285b55332c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:51.288807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:51.290191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:51.295519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 994/1964] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:51.306901 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bba7e-d1ce-462b-b517-9c3e5054ce7d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:51.308748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6d4bba7e-d1ce-462b-b517-9c3e5054ce7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:51.360968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3649f8-2a63-4dc0-9dde-3b998b156a8c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:51.365864 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:51.366499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:51.366804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:51.367043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] PATH_INFO: `/v3/users/e35161d7984d44bbaaa5e0f43482da09` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:51.368177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35161d7984d44bbaaa5e0f43482da09)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:51.370138 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:51.377020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bba7e-d1ce-462b-b517-9c3e5054ce7d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:51.381606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90782051-bed6-49a6-b4f9-e49a43fe1751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:51.382250 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90782051-bed6-49a6-b4f9-e49a43fe1751 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:51.382530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90782051-bed6-49a6-b4f9-e49a43fe1751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:51.382809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90782051-bed6-49a6-b4f9-e49a43fe1751 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:51.383468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90782051-bed6-49a6-b4f9-e49a43fe1751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:51.385392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90782051-bed6-49a6-b4f9-e49a43fe1751 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:51.474238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 995/1965] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:51 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:51.475999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90782051-bed6-49a6-b4f9-e49a43fe1751 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:51.477573 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-90782051-bed6-49a6-b4f9-e49a43fe1751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:51.543468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90782051-bed6-49a6-b4f9-e49a43fe1751 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:51.549631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fa166-ab7b-4c7e-89a6-061b0845cb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:51.550292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fa166-ab7b-4c7e-89a6-061b0845cb38 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:51.550573 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fa166-ab7b-4c7e-89a6-061b0845cb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:51.550848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fa166-ab7b-4c7e-89a6-061b0845cb38 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:51.550976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35161d7984d44bbaaa5e0f43482da09'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:51.551493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fa166-ab7b-4c7e-89a6-061b0845cb38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:51.553416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fa166-ab7b-4c7e-89a6-061b0845cb38 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:51.663632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35161d7984d44bbaaa5e0f43482da09'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:51.664296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35161d7984d44bbaaa5e0f43482da09'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:51.671768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35161d7984d44bbaaa5e0f43482da09'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:51.683799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35161d7984d44bbaaa5e0f43482da09'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:51.690894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 971/1966] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:51 2022] DELETE /identity/v3/users/e35161d7984d44bbaaa5e0f43482da09 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:51.715733 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 996/1967] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:51 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:51.720080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:51.722065 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:51.782219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:51.830439 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a0a0c-4c99-4ad4-a1d6-c682a51fe7ca admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:51.835691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb4364-0c36-4d2a-b614-dcbb6f1539a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:51.836314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb4364-0c36-4d2a-b614-dcbb6f1539a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:51.836590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb4364-0c36-4d2a-b614-dcbb6f1539a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:51.837116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb4364-0c36-4d2a-b614-dcbb6f1539a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:51.838029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb4364-0c36-4d2a-b614-dcbb6f1539a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:51.839911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb4364-0c36-4d2a-b614-dcbb6f1539a4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:51.856038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 972/1968] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:51.860542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fa166-ab7b-4c7e-89a6-061b0845cb38 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:51.869187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fa166-ab7b-4c7e-89a6-061b0845cb38 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:51.874176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928dcc6-d64f-44b8-86e9-4fa4aac94d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:51.875590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928dcc6-d64f-44b8-86e9-4fa4aac94d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:51.876085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928dcc6-d64f-44b8-86e9-4fa4aac94d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:51.876416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928dcc6-d64f-44b8-86e9-4fa4aac94d93 admin admin] PATH_INFO: `/v3/projects/9c9423d36b6540f99953a4598fa4feea/users/8a4e89ad2e3a497b88c9ad0da1396b6d/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:51.894884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928dcc6-d64f-44b8-86e9-4fa4aac94d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9423d36b6540f99953a4598fa4feea, user_id=8a4e89ad2e3a497b88c9ad0da1396b6d, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:51.897087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928dcc6-d64f-44b8-86e9-4fa4aac94d93 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:52.013124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 997/1969] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:51 2022] PUT /identity/v3/projects/9c9423d36b6540f99953a4598fa4feea/users/8a4e89ad2e3a497b88c9ad0da1396b6d/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:52.018403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb4364-0c36-4d2a-b614-dcbb6f1539a4 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:52.034053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb4364-0c36-4d2a-b614-dcbb6f1539a4 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:52.105285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2bb6e-149e-4950-91bf-fd91ee8a703d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:52.105947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2bb6e-149e-4950-91bf-fd91ee8a703d admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:52.106253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2bb6e-149e-4950-91bf-fd91ee8a703d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:52.106693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2bb6e-149e-4950-91bf-fd91ee8a703d admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:52.107365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2bb6e-149e-4950-91bf-fd91ee8a703d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:52.109223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2bb6e-149e-4950-91bf-fd91ee8a703d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:52.122769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 973/1970] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:52.123963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928dcc6-d64f-44b8-86e9-4fa4aac94d93 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:52.134791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928dcc6-d64f-44b8-86e9-4fa4aac94d93 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:52.150633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7842a0-3878-423c-a309-126fa132a4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:52.151455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7842a0-3878-423c-a309-126fa132a4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:52.151802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7842a0-3878-423c-a309-126fa132a4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:52.152184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7842a0-3878-423c-a309-126fa132a4d4 admin admin] PATH_INFO: `/v3/projects/9c9423d36b6540f99953a4598fa4feea/users/8a4e89ad2e3a497b88c9ad0da1396b6d/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:52.156620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7842a0-3878-423c-a309-126fa132a4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9423d36b6540f99953a4598fa4feea, user_id=8a4e89ad2e3a497b88c9ad0da1396b6d, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:52.160003 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7842a0-3878-423c-a309-126fa132a4d4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:52.173812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 998/1971] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:52 2022] PUT /identity/v3/projects/9c9423d36b6540f99953a4598fa4feea/users/8a4e89ad2e3a497b88c9ad0da1396b6d/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:52.384193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2bb6e-149e-4950-91bf-fd91ee8a703d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:52.386202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-3ec2bb6e-149e-4950-91bf-fd91ee8a703d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:52.399649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2bb6e-149e-4950-91bf-fd91ee8a703d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:52.467171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:52.468397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:52.468397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:52.468654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] PATH_INFO: `/v3/projects/eb7b75093a2047ac91ae7a92bb5c2a6f` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:52.469856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb7b75093a2047ac91ae7a92bb5c2a6f)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:52.471911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:52.641042 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb7b75093a2047ac91ae7a92bb5c2a6f', 'request_id': 'req-e3efb2ed-a09d-4f57-888d-925ec927c8d0'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:52.715017 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb7b75093a2047ac91ae7a92bb5c2a6f', 'request_id': 'req-e3efb2ed-a09d-4f57-888d-925ec927c8d0'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:52.721432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb7b75093a2047ac91ae7a92bb5c2a6f', 'request_id': 'req-e3efb2ed-a09d-4f57-888d-925ec927c8d0'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:52.733409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] The token cache is being invalidate because project eb7b75093a2047ac91ae7a92bb5c2a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:52.733692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:52.734913 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:52.736448 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 974/1972] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:52 2022] DELETE /identity/v3/projects/eb7b75093a2047ac91ae7a92bb5c2a6f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:52.740547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7842a0-3878-423c-a309-126fa132a4d4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:52.742226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-de7842a0-3878-423c-a309-126fa132a4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:52.755348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7842a0-3878-423c-a309-126fa132a4d4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:52.813421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:52.814108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:52.814387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:52.814659 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] PATH_INFO: `/v3/users/661d2d26882a48c7944150217022c131` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:52.815766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] RBAC: Authorizing `identity:delete_user(user_id=661d2d26882a48c7944150217022c131)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:52.817734 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:53.058993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661d2d26882a48c7944150217022c131'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:53.071835 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661d2d26882a48c7944150217022c131'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:53.200452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661d2d26882a48c7944150217022c131'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:53.201326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661d2d26882a48c7944150217022c131'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:53.209953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-924fbb9f-2209-4b88-b7f7-1c479793a12c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661d2d26882a48c7944150217022c131'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:53.216946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 999/1973] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:52 2022] DELETE /identity/v3/users/661d2d26882a48c7944150217022c131 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:53.715984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:53.726626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efb2ed-a09d-4f57-888d-925ec927c8d0 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:53.731722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:53.791727 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:53.792467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:53.792809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:53.793118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] PATH_INFO: `/v3/projects/2b0c1472465e47daa3d38a2a93ced10f` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:53.794625 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b0c1472465e47daa3d38a2a93ced10f)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:53.796761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:53.912904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:53.929512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b0c1472465e47daa3d38a2a93ced10f', 'request_id': 'req-ce8fb5c3-3588-4b7e-a014-4894accffd2d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:54.027428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b0c1472465e47daa3d38a2a93ced10f', 'request_id': 'req-ce8fb5c3-3588-4b7e-a014-4894accffd2d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:54.034396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b0c1472465e47daa3d38a2a93ced10f', 'request_id': 'req-ce8fb5c3-3588-4b7e-a014-4894accffd2d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:54.058226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] The token cache is being invalidate because project 2b0c1472465e47daa3d38a2a93ced10f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:54.058590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:54.059316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ce8fb5c3-3588-4b7e-a014-4894accffd2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:54.062975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 975/1974] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:53 2022] DELETE /identity/v3/projects/2b0c1472465e47daa3d38a2a93ced10f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:54.070037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06ecdc-b69d-4eed-8d7b-68b760e1a7fe None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.070512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06ecdc-b69d-4eed-8d7b-68b760e1a7fe None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.070932 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06ecdc-b69d-4eed-8d7b-68b760e1a7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.081253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2a6c-f90c-4bf8-82e5-d95a56dbd2dd None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.081529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2a6c-f90c-4bf8-82e5-d95a56dbd2dd None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.081834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2a6c-f90c-4bf8-82e5-d95a56dbd2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.097197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0d06ecdc-b69d-4eed-8d7b-68b760e1a7fe None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:54.098904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0d06ecdc-b69d-4eed-8d7b-68b760e1a7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:54.113029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-813f2a6c-f90c-4bf8-82e5-d95a56dbd2dd None None] MFA Rules not processed for user `8a4e89ad2e3a497b88c9ad0da1396b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:54.114426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-813f2a6c-f90c-4bf8-82e5-d95a56dbd2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:54.176603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1000/1975] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:54.180706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aa5a1-2976-49e9-9965-7a6233f423d2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.180994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aa5a1-2976-49e9-9965-7a6233f423d2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.181746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aa5a1-2976-49e9-9965-7a6233f423d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.189478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 976/1976] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:54.201557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f2a6c-f90c-4bf8-82e5-d95a56dbd2dd None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:54.202991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-813f2a6c-f90c-4bf8-82e5-d95a56dbd2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:54.203107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a18aa5a1-2976-49e9-9965-7a6233f423d2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:54.204194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a18aa5a1-2976-49e9-9965-7a6233f423d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:54.212695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1001/1977] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:54.216737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aa5a1-2976-49e9-9965-7a6233f423d2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:54.226262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aa5a1-2976-49e9-9965-7a6233f423d2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:54.231360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1885573-fbfa-4324-87f3-a39c1fa9702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:54.231970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1885573-fbfa-4324-87f3-a39c1fa9702c admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.232248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1885573-fbfa-4324-87f3-a39c1fa9702c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.232522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1885573-fbfa-4324-87f3-a39c1fa9702c admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.233176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1885573-fbfa-4324-87f3-a39c1fa9702c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:54.233569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1885573-fbfa-4324-87f3-a39c1fa9702c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:54.235887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1885573-fbfa-4324-87f3-a39c1fa9702c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:54.247815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1002/1978] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:49:54.252735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1885573-fbfa-4324-87f3-a39c1fa9702c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:54.262433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1885573-fbfa-4324-87f3-a39c1fa9702c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:54.266498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f2a6c-f90c-4bf8-82e5-d95a56dbd2dd None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:54.267203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3462e3b-ff91-40b0-ab18-ec8083979b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:54.267793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3462e3b-ff91-40b0-ab18-ec8083979b00 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.268069 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3462e3b-ff91-40b0-ab18-ec8083979b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.268394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3462e3b-ff91-40b0-ab18-ec8083979b00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.268965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3462e3b-ff91-40b0-ab18-ec8083979b00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:54.270917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3462e3b-ff91-40b0-ab18-ec8083979b00 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:54.271226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cab566-3db7-4074-ad4d-735a7aff4f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:54.271856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab566-3db7-4074-ad4d-735a7aff4f5d service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.272134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab566-3db7-4074-ad4d-735a7aff4f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.280164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab566-3db7-4074-ad4d-735a7aff4f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.280164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b1cab566-3db7-4074-ad4d-735a7aff4f5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:54.335716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cab566-3db7-4074-ad4d-735a7aff4f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:54.337409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cab566-3db7-4074-ad4d-735a7aff4f5d service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:54.346202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 977/1979] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:54.412057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1003/1980] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:54 2022] POST /identity/v3/projects => generated 409 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:54.416442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cab566-3db7-4074-ad4d-735a7aff4f5d service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:54.428692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cab566-3db7-4074-ad4d-735a7aff4f5d service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:54.507154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573bce7-bbe7-4041-a420-9b7c555af19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:54.508048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573bce7-bbe7-4041-a420-9b7c555af19e admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.508489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573bce7-bbe7-4041-a420-9b7c555af19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.508905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573bce7-bbe7-4041-a420-9b7c555af19e admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.509581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573bce7-bbe7-4041-a420-9b7c555af19e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:54.511818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573bce7-bbe7-4041-a420-9b7c555af19e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:54.670421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 978/1981] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:54 2022] POST /identity/v3/users => generated 305 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:54.675167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3462e3b-ff91-40b0-ab18-ec8083979b00 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:54.685918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3462e3b-ff91-40b0-ab18-ec8083979b00 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:54.686054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573bce7-bbe7-4041-a420-9b7c555af19e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:54.687774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b573bce7-bbe7-4041-a420-9b7c555af19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:54.690689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8410c-b39b-4b5b-acf8-29668f16ed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:54.691432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8410c-b39b-4b5b-acf8-29668f16ed29 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.691716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8410c-b39b-4b5b-acf8-29668f16ed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.691992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8410c-b39b-4b5b-acf8-29668f16ed29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.692746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8410c-b39b-4b5b-acf8-29668f16ed29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:54.694374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8410c-b39b-4b5b-acf8-29668f16ed29 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:54.699981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573bce7-bbe7-4041-a420-9b7c555af19e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:54.706083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1004/1982] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:54.706215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:54.706602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.706952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.707295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] PATH_INFO: `/v3/users/96e63c137b4d4c66a983f54a39fa0b01` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.708696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e63c137b4d4c66a983f54a39fa0b01)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:54.708869 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8410c-b39b-4b5b-acf8-29668f16ed29 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:54.710658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:54.719450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8410c-b39b-4b5b-acf8-29668f16ed29 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:54.723888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a4771-98b9-4d30-bdca-34727c7f0f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:54.724470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a4771-98b9-4d30-bdca-34727c7f0f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.724844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a4771-98b9-4d30-bdca-34727c7f0f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.725054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a4771-98b9-4d30-bdca-34727c7f0f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.725682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a4771-98b9-4d30-bdca-34727c7f0f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:54.727290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a4771-98b9-4d30-bdca-34727c7f0f56 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:54.738940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1005/1983] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:49:54.743180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a4771-98b9-4d30-bdca-34727c7f0f56 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:54.753384 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a4771-98b9-4d30-bdca-34727c7f0f56 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:54.760215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2d493-91d7-4199-a02c-78d1c42dee08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:54.760215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2d493-91d7-4199-a02c-78d1c42dee08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.760215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2d493-91d7-4199-a02c-78d1c42dee08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.760215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2d493-91d7-4199-a02c-78d1c42dee08 admin admin] PATH_INFO: `/v3/projects/38891ab9e3014b67b8f431ac9b4b1140/users/4d1d51e9c1b44783b5c5931fa2e4ef72/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.773930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2d493-91d7-4199-a02c-78d1c42dee08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38891ab9e3014b67b8f431ac9b4b1140, user_id=4d1d51e9c1b44783b5c5931fa2e4ef72, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:54.776549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2d493-91d7-4199-a02c-78d1c42dee08 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:54.895166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1006/1984] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:54 2022] PUT /identity/v3/projects/38891ab9e3014b67b8f431ac9b4b1140/users/4d1d51e9c1b44783b5c5931fa2e4ef72/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:54.906910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24d73-f30e-4ed0-b48a-6542806b32ea None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:54.907200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24d73-f30e-4ed0-b48a-6542806b32ea None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:54.907480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24d73-f30e-4ed0-b48a-6542806b32ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:54.936563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-12c24d73-f30e-4ed0-b48a-6542806b32ea None None] MFA Rules not processed for user `4d1d51e9c1b44783b5c5931fa2e4ef72`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:54.938695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-12c24d73-f30e-4ed0-b48a-6542806b32ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:54.960446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e63c137b4d4c66a983f54a39fa0b01'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:55.013816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1007/1985] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:55.018899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c24d73-f30e-4ed0-b48a-6542806b32ea None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:55.034605 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e63c137b4d4c66a983f54a39fa0b01'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:55.035272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e63c137b4d4c66a983f54a39fa0b01'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:55.042880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e63c137b4d4c66a983f54a39fa0b01'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:55.055811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-036f9f44-f216-4077-9866-d914e2958a70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e63c137b4d4c66a983f54a39fa0b01'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:55.061844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 979/1986] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:54 2022] DELETE /identity/v3/users/96e63c137b4d4c66a983f54a39fa0b01 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:55.066452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13f3c4-f2d1-4df9-ae6f-3a532d7f4867 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:55.066887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13f3c4-f2d1-4df9-ae6f-3a532d7f4867 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:55.067310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13f3c4-f2d1-4df9-ae6f-3a532d7f4867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:55.084324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c24d73-f30e-4ed0-b48a-6542806b32ea None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:55.094069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-cf13f3c4-f2d1-4df9-ae6f-3a532d7f4867 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:55.108738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-cf13f3c4-f2d1-4df9-ae6f-3a532d7f4867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:55.148479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a498da6-55ce-496e-bd16-a3dbaa28c14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:55.149096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a498da6-55ce-496e-bd16-a3dbaa28c14b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:55.149579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a498da6-55ce-496e-bd16-a3dbaa28c14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:55.149897 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a498da6-55ce-496e-bd16-a3dbaa28c14b admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:55.150546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a498da6-55ce-496e-bd16-a3dbaa28c14b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:55.152698 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a498da6-55ce-496e-bd16-a3dbaa28c14b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:55.179858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 980/1987] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:55.193045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13f3c4-f2d1-4df9-ae6f-3a532d7f4867 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:55.200222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-cf13f3c4-f2d1-4df9-ae6f-3a532d7f4867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:55.254237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:55.265602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13f3c4-f2d1-4df9-ae6f-3a532d7f4867 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:55.269291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715a78c-e31c-459d-800a-525f855b5b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:55.269931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715a78c-e31c-459d-800a-525f855b5b2a service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:55.270206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715a78c-e31c-459d-800a-525f855b5b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:55.270473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715a78c-e31c-459d-800a-525f855b5b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:55.279936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715a78c-e31c-459d-800a-525f855b5b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:55.282607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715a78c-e31c-459d-800a-525f855b5b2a service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:55.293044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 981/1988] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:55.294623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1008/1989] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:55 2022] POST /identity/v3/projects => generated 405 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:55.298825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715a78c-e31c-459d-800a-525f855b5b2a service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:55.300246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6715a78c-e31c-459d-800a-525f855b5b2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:55.314103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715a78c-e31c-459d-800a-525f855b5b2a service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:55.372926 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4e57-c4fd-498c-860f-3265b66c0463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:55.373536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4e57-c4fd-498c-860f-3265b66c0463 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:55.373850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4e57-c4fd-498c-860f-3265b66c0463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:55.374140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4e57-c4fd-498c-860f-3265b66c0463 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:55.375312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b4e57-c4fd-498c-860f-3265b66c0463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:55.377419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b4e57-c4fd-498c-860f-3265b66c0463 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:55.585418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 982/1990] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:55 2022] POST /identity/v3/users => generated 309 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:55.586816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a498da6-55ce-496e-bd16-a3dbaa28c14b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:55.597847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a498da6-55ce-496e-bd16-a3dbaa28c14b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:55.602467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081a985-d8dd-426b-ba5c-73880dfbe2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:55.603140 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081a985-d8dd-426b-ba5c-73880dfbe2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:55.603515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081a985-d8dd-426b-ba5c-73880dfbe2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:55.603874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081a985-d8dd-426b-ba5c-73880dfbe2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:55.604581 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081a985-d8dd-426b-ba5c-73880dfbe2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:55.606285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081a985-d8dd-426b-ba5c-73880dfbe2d3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:55.618450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1009/1991] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:55.622759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4e57-c4fd-498c-860f-3265b66c0463 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:55.645204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4e57-c4fd-498c-860f-3265b66c0463 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:55.650604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaaede1-8f35-4e8f-a6e6-4a1cffe54762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:55.651269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaede1-8f35-4e8f-a6e6-4a1cffe54762 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:55.651553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaede1-8f35-4e8f-a6e6-4a1cffe54762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:55.651829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaede1-8f35-4e8f-a6e6-4a1cffe54762 admin admin] PATH_INFO: `/v3/projects/0202b7c6ba6348d68b74c188c3afa5ca/users/de8964b856be49eaadfe7a1e97ba3c16/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:55.671947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaaede1-8f35-4e8f-a6e6-4a1cffe54762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0202b7c6ba6348d68b74c188c3afa5ca, user_id=de8964b856be49eaadfe7a1e97ba3c16, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:55.674667 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaaede1-8f35-4e8f-a6e6-4a1cffe54762 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:55.783451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 983/1992] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:55 2022] PUT /identity/v3/projects/0202b7c6ba6348d68b74c188c3afa5ca/users/de8964b856be49eaadfe7a1e97ba3c16/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:55.784878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081a985-d8dd-426b-ba5c-73880dfbe2d3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:55.797434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081a985-d8dd-426b-ba5c-73880dfbe2d3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:55.866492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc16ffb-602a-4dcf-a538-a4f6a9e65893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:55.867161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc16ffb-602a-4dcf-a538-a4f6a9e65893 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:55.867438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc16ffb-602a-4dcf-a538-a4f6a9e65893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:55.867853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc16ffb-602a-4dcf-a538-a4f6a9e65893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:55.868616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc16ffb-602a-4dcf-a538-a4f6a9e65893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:55.870233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc16ffb-602a-4dcf-a538-a4f6a9e65893 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:55.889906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1010/1993] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:55.889906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc16ffb-602a-4dcf-a538-a4f6a9e65893 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:55.890965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaaede1-8f35-4e8f-a6e6-4a1cffe54762 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:55.890965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-cfaaede1-8f35-4e8f-a6e6-4a1cffe54762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:55.898672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc16ffb-602a-4dcf-a538-a4f6a9e65893 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:55.902073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaaede1-8f35-4e8f-a6e6-4a1cffe54762 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:55.904408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510b2ab-1f66-4891-8fd6-847881ba865d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:55.905088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510b2ab-1f66-4891-8fd6-847881ba865d admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:55.905440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510b2ab-1f66-4891-8fd6-847881ba865d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:55.905792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510b2ab-1f66-4891-8fd6-847881ba865d admin admin] PATH_INFO: `/v3/projects/0202b7c6ba6348d68b74c188c3afa5ca/users/de8964b856be49eaadfe7a1e97ba3c16/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:55.908598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510b2ab-1f66-4891-8fd6-847881ba865d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0202b7c6ba6348d68b74c188c3afa5ca, user_id=de8964b856be49eaadfe7a1e97ba3c16, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:55.909682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:55.912508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:55.912508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:55.912508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] PATH_INFO: `/v3/projects/00eeab850ffb470eaa58033f7e5f95e0` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:55.912833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510b2ab-1f66-4891-8fd6-847881ba865d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:55.912924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00eeab850ffb470eaa58033f7e5f95e0)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:55.914213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:55.925304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1011/1994] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:55 2022] PUT /identity/v3/projects/0202b7c6ba6348d68b74c188c3afa5ca/users/de8964b856be49eaadfe7a1e97ba3c16/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:55.937719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe441a1-cd5a-4bc1-be43-e78feacbdca5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:55.937719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe441a1-cd5a-4bc1-be43-e78feacbdca5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:55.937719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe441a1-cd5a-4bc1-be43-e78feacbdca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:55.966771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ebe441a1-cd5a-4bc1-be43-e78feacbdca5 None None] MFA Rules not processed for user `de8964b856be49eaadfe7a1e97ba3c16`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:55.968191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ebe441a1-cd5a-4bc1-be43-e78feacbdca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:56.039435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1012/1995] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:55 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:56.051129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe441a1-cd5a-4bc1-be43-e78feacbdca5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:56.052465 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ebe441a1-cd5a-4bc1-be43-e78feacbdca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:56.068924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00eeab850ffb470eaa58033f7e5f95e0', 'request_id': 'req-35fbbf39-540c-4ae1-a658-5997e866fbb9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:56.110555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe441a1-cd5a-4bc1-be43-e78feacbdca5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:56.158490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309980-6b23-489e-81b0-f37a43f6884b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:56.159105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309980-6b23-489e-81b0-f37a43f6884b service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:56.159382 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309980-6b23-489e-81b0-f37a43f6884b service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:56.159653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309980-6b23-489e-81b0-f37a43f6884b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:56.223492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0309980-6b23-489e-81b0-f37a43f6884b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:56.224948 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0309980-6b23-489e-81b0-f37a43f6884b service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:56.232336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00eeab850ffb470eaa58033f7e5f95e0', 'request_id': 'req-35fbbf39-540c-4ae1-a658-5997e866fbb9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:56.235397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1013/1996] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:56 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:56.241081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00eeab850ffb470eaa58033f7e5f95e0', 'request_id': 'req-35fbbf39-540c-4ae1-a658-5997e866fbb9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:56.252930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] The token cache is being invalidate because project 00eeab850ffb470eaa58033f7e5f95e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:56.253444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:56.254273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:56.256086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 984/1997] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:55 2022] DELETE /identity/v3/projects/00eeab850ffb470eaa58033f7e5f95e0 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:49:56.260114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309980-6b23-489e-81b0-f37a43f6884b service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:56.334017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309980-6b23-489e-81b0-f37a43f6884b service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:56.339142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:56.339805 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:56.340119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:56.340401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] PATH_INFO: `/v3/users/d35bd3b489124162a42a64f5ab3a88b3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:56.341536 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35bd3b489124162a42a64f5ab3a88b3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:56.344522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:56.448439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:56.574437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35bd3b489124162a42a64f5ab3a88b3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:56.578106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:56.580823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:56.588372 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35bd3b489124162a42a64f5ab3a88b3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:56.594578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbbf39-540c-4ae1-a658-5997e866fbb9 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:56.598262 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb1e9-7706-442a-bf1e-5b8284fe6dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:56.598893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb1e9-7706-442a-bf1e-5b8284fe6dfd service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:56.599173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb1e9-7706-442a-bf1e-5b8284fe6dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:56.599450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb1e9-7706-442a-bf1e-5b8284fe6dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:56.601165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-299fb1e9-7706-442a-bf1e-5b8284fe6dfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:56.665503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb1e9-7706-442a-bf1e-5b8284fe6dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:56.666959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb1e9-7706-442a-bf1e-5b8284fe6dfd service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:56.672532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35bd3b489124162a42a64f5ab3a88b3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:56.673186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35bd3b489124162a42a64f5ab3a88b3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:56.681247 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35bd3b489124162a42a64f5ab3a88b3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:56.686917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 985/1998] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:56.689243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1014/1999] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:49:56 2022] DELETE /identity/v3/users/d35bd3b489124162a42a64f5ab3a88b3 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:56.753141 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb1e9-7706-442a-bf1e-5b8284fe6dfd service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:56.755020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-299fb1e9-7706-442a-bf1e-5b8284fe6dfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:56.769840 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb1e9-7706-442a-bf1e-5b8284fe6dfd service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:56.837917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938af36b-cfce-4bba-ae90-3661cd310e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:56.838536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938af36b-cfce-4bba-ae90-3661cd310e9e service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:56.838818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938af36b-cfce-4bba-ae90-3661cd310e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:56.839090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938af36b-cfce-4bba-ae90-3661cd310e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:56.840883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-938af36b-cfce-4bba-ae90-3661cd310e9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:56.910738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938af36b-cfce-4bba-ae90-3661cd310e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:56.912182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938af36b-cfce-4bba-ae90-3661cd310e9e service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:56.922550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 986/2000] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:56 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:49:57.245479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:57.248970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:57.346449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ce915-6f71-4ea0-b42f-1147c3652f20 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:57.351663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:57.352307 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:57.352591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:57.352917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] PATH_INFO: `/v3/projects/cf5a9563a1d647d781f94379efc9d148` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:57.354078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5a9563a1d647d781f94379efc9d148)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:57.356053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:57.463741 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5a9563a1d647d781f94379efc9d148', 'request_id': 'req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:57.470493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5a9563a1d647d781f94379efc9d148', 'request_id': 'req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:57.563918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5a9563a1d647d781f94379efc9d148', 'request_id': 'req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:57.576461 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] The token cache is being invalidate because project cf5a9563a1d647d781f94379efc9d148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:49:57.576760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:57.577425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c78b6df8-3066-4ec2-b1ad-e61a881d11e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:49:57.579261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1015/2001] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:49:57 2022] DELETE /identity/v3/projects/cf5a9563a1d647d781f94379efc9d148 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:57.584407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ddfcd-b333-4063-bc8e-318a0ee655c2 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:57.584881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ddfcd-b333-4063-bc8e-318a0ee655c2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:57.585035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ddfcd-b333-4063-bc8e-318a0ee655c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:57.606807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-953ddfcd-b333-4063-bc8e-318a0ee655c2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:57.608025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-953ddfcd-b333-4063-bc8e-318a0ee655c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:57.670112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 987/2002] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:49:57.678758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8348f8-ec47-40e9-b5cd-87358806ae0b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:57.679047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8348f8-ec47-40e9-b5cd-87358806ae0b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:57.679322 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8348f8-ec47-40e9-b5cd-87358806ae0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:57.718626 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9f8348f8-ec47-40e9-b5cd-87358806ae0b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:57.718626 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9f8348f8-ec47-40e9-b5cd-87358806ae0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:57.725061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1016/2003] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:57.729479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ddfcd-b333-4063-bc8e-318a0ee655c2 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:57.740594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ddfcd-b333-4063-bc8e-318a0ee655c2 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:57.753289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7be138-6896-41b6-ae3d-b3cf395a3c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:57.753289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7be138-6896-41b6-ae3d-b3cf395a3c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:57.753289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7be138-6896-41b6-ae3d-b3cf395a3c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:57.753289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7be138-6896-41b6-ae3d-b3cf395a3c5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:57.753620 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7be138-6896-41b6-ae3d-b3cf395a3c5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:49:57.753620 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7be138-6896-41b6-ae3d-b3cf395a3c5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:57.753620 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7be138-6896-41b6-ae3d-b3cf395a3c5f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:57.761665 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 988/2004] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:49:57.766921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8348f8-ec47-40e9-b5cd-87358806ae0b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:57.776261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8348f8-ec47-40e9-b5cd-87358806ae0b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:57.781646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2512c6-a26a-4b78-88f2-24254a94dd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:57.781646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2512c6-a26a-4b78-88f2-24254a94dd80 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:57.781997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2512c6-a26a-4b78-88f2-24254a94dd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:57.782173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2512c6-a26a-4b78-88f2-24254a94dd80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:57.782800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2512c6-a26a-4b78-88f2-24254a94dd80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:57.784683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2512c6-a26a-4b78-88f2-24254a94dd80 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:57.891117 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:57.898499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1017/2005] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:57 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:49:57.902560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7be138-6896-41b6-ae3d-b3cf395a3c5f admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:57.912851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7be138-6896-41b6-ae3d-b3cf395a3c5f admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:57.971593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef750296-9c79-4154-a506-999b2b68dda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:57.972126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef750296-9c79-4154-a506-999b2b68dda4 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:57.972408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef750296-9c79-4154-a506-999b2b68dda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:57.972682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef750296-9c79-4154-a506-999b2b68dda4 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:57.973330 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef750296-9c79-4154-a506-999b2b68dda4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:57.975379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef750296-9c79-4154-a506-999b2b68dda4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:58.250973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 989/2006] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:57 2022] POST /identity/v3/users => generated 283 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:58.258863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2512c6-a26a-4b78-88f2-24254a94dd80 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:58.262818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2512c6-a26a-4b78-88f2-24254a94dd80 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:58.268114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b6cc5-6e52-4a2e-8063-5c609b7d4f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:58.268717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b6cc5-6e52-4a2e-8063-5c609b7d4f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:58.268997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b6cc5-6e52-4a2e-8063-5c609b7d4f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:58.269331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b6cc5-6e52-4a2e-8063-5c609b7d4f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:58.269991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b6cc5-6e52-4a2e-8063-5c609b7d4f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:58.271573 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b6cc5-6e52-4a2e-8063-5c609b7d4f02 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:58.282214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1018/2007] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:58.286214 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef750296-9c79-4154-a506-999b2b68dda4 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:58.297510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef750296-9c79-4154-a506-999b2b68dda4 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:58.302045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f92be-32b3-4d34-b182-8e1eaa93a8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:58.302706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f92be-32b3-4d34-b182-8e1eaa93a8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:58.303008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f92be-32b3-4d34-b182-8e1eaa93a8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:58.303295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f92be-32b3-4d34-b182-8e1eaa93a8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:58.303928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f92be-32b3-4d34-b182-8e1eaa93a8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:58.305522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f92be-32b3-4d34-b182-8e1eaa93a8b9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:58.316193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 990/2008] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:58.320438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b6cc5-6e52-4a2e-8063-5c609b7d4f02 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:58.330532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b6cc5-6e52-4a2e-8063-5c609b7d4f02 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:58.335405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02001f21-071c-4154-bd68-dec298a00504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:58.336067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02001f21-071c-4154-bd68-dec298a00504 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:58.336438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02001f21-071c-4154-bd68-dec298a00504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:58.336624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02001f21-071c-4154-bd68-dec298a00504 admin admin] PATH_INFO: `/v3/projects/1a573da49b7f4c799e51cc4115314f35/users/e318fcfe071c47e5a35165a10828b55d/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:58.351743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02001f21-071c-4154-bd68-dec298a00504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a573da49b7f4c799e51cc4115314f35, user_id=e318fcfe071c47e5a35165a10828b55d, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:58.354444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02001f21-071c-4154-bd68-dec298a00504 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:58.448134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1019/2009] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:58 2022] PUT /identity/v3/projects/1a573da49b7f4c799e51cc4115314f35/users/e318fcfe071c47e5a35165a10828b55d/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:58.459587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1382b3b-2f2b-4c01-9fcd-ea2124626d0d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:58.459872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1382b3b-2f2b-4c01-9fcd-ea2124626d0d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:58.460145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1382b3b-2f2b-4c01-9fcd-ea2124626d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:58.490199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b1382b3b-2f2b-4c01-9fcd-ea2124626d0d None None] MFA Rules not processed for user `e318fcfe071c47e5a35165a10828b55d`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:58.491537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b1382b3b-2f2b-4c01-9fcd-ea2124626d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:58.564827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 991/2010] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:58.569526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02001f21-071c-4154-bd68-dec298a00504 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:58.680923 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02001f21-071c-4154-bd68-dec298a00504 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:58.687671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e676f8-53f7-4158-b600-764be3e52aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:58.690423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e676f8-53f7-4158-b600-764be3e52aed admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:58.690728 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e676f8-53f7-4158-b600-764be3e52aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:58.691019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e676f8-53f7-4158-b600-764be3e52aed admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:58.692957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e676f8-53f7-4158-b600-764be3e52aed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:58.695223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e676f8-53f7-4158-b600-764be3e52aed admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:58.836695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1020/2011] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:49:58 2022] POST /identity/v3/projects => generated 365 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:58.847348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1382b3b-2f2b-4c01-9fcd-ea2124626d0d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:58.858497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1382b3b-2f2b-4c01-9fcd-ea2124626d0d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:58.863650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:49:58.928973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d247ed-bcee-49c2-97de-3a394746eb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:58.929595 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d247ed-bcee-49c2-97de-3a394746eb4b admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:58.929909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d247ed-bcee-49c2-97de-3a394746eb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:58.930192 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d247ed-bcee-49c2-97de-3a394746eb4b admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:58.930847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d247ed-bcee-49c2-97de-3a394746eb4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:58.932807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d247ed-bcee-49c2-97de-3a394746eb4b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:59.127152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 992/2012] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:49:58 2022] POST /identity/v3/users => generated 289 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:49:59.134492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e676f8-53f7-4158-b600-764be3e52aed admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:59.149173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e676f8-53f7-4158-b600-764be3e52aed admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:59.154709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa602b08-78af-4d35-87f9-2e869c17fe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:59.157999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa602b08-78af-4d35-87f9-2e869c17fe31 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:59.157999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa602b08-78af-4d35-87f9-2e869c17fe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:59.157999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa602b08-78af-4d35-87f9-2e869c17fe31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:59.158277 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa602b08-78af-4d35-87f9-2e869c17fe31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:59.161026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa602b08-78af-4d35-87f9-2e869c17fe31 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:59.173138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1021/2013] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:59.177799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d247ed-bcee-49c2-97de-3a394746eb4b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:59.189144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d247ed-bcee-49c2-97de-3a394746eb4b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:59.193975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef44161-8253-4699-8f8d-e368dff3273b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:59.203907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef44161-8253-4699-8f8d-e368dff3273b admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:59.203907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef44161-8253-4699-8f8d-e368dff3273b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:59.203907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef44161-8253-4699-8f8d-e368dff3273b admin admin] PATH_INFO: `/v3/projects/2e26e3c8ed40450489075a614b0d4465/users/2ce7a9bfd8ed46f2a0d370e2dbff7c8b/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:59.216599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef44161-8253-4699-8f8d-e368dff3273b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e26e3c8ed40450489075a614b0d4465, user_id=2ce7a9bfd8ed46f2a0d370e2dbff7c8b, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:59.220932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef44161-8253-4699-8f8d-e368dff3273b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:59.424447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 993/2014] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:59 2022] PUT /identity/v3/projects/2e26e3c8ed40450489075a614b0d4465/users/2ce7a9bfd8ed46f2a0d370e2dbff7c8b/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:59.425680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa602b08-78af-4d35-87f9-2e869c17fe31 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:59.436535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa602b08-78af-4d35-87f9-2e869c17fe31 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:59.523797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4338f-b66a-4ecb-8ca7-4b3ffc408e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:59.524405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4338f-b66a-4ecb-8ca7-4b3ffc408e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:59.524681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4338f-b66a-4ecb-8ca7-4b3ffc408e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:59.524953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4338f-b66a-4ecb-8ca7-4b3ffc408e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:59.525628 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4338f-b66a-4ecb-8ca7-4b3ffc408e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:59.527220 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4338f-b66a-4ecb-8ca7-4b3ffc408e57 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:59.538124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1022/2015] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:49:59.542676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef44161-8253-4699-8f8d-e368dff3273b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:59.553770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef44161-8253-4699-8f8d-e368dff3273b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:59.558436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39845e-8a8a-4511-ac83-015b413b52d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:59.559091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39845e-8a8a-4511-ac83-015b413b52d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:59.559457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39845e-8a8a-4511-ac83-015b413b52d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:59.559734 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39845e-8a8a-4511-ac83-015b413b52d3 admin admin] PATH_INFO: `/v3/projects/2e26e3c8ed40450489075a614b0d4465/users/2ce7a9bfd8ed46f2a0d370e2dbff7c8b/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:59.561718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39845e-8a8a-4511-ac83-015b413b52d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e26e3c8ed40450489075a614b0d4465, user_id=2ce7a9bfd8ed46f2a0d370e2dbff7c8b, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:59.565066 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39845e-8a8a-4511-ac83-015b413b52d3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:59.575825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 994/2016] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:49:59 2022] PUT /identity/v3/projects/2e26e3c8ed40450489075a614b0d4465/users/2ce7a9bfd8ed46f2a0d370e2dbff7c8b/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:49:59.587943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bcb6f-65a4-48e6-9445-8b8dad72caa2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:59.588234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bcb6f-65a4-48e6-9445-8b8dad72caa2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:59.588514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bcb6f-65a4-48e6-9445-8b8dad72caa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:59.623859 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c09bcb6f-65a4-48e6-9445-8b8dad72caa2 None None] MFA Rules not processed for user `2ce7a9bfd8ed46f2a0d370e2dbff7c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:49:59.623859 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c09bcb6f-65a4-48e6-9445-8b8dad72caa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:59.695209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1023/2017] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:49:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:49:59.702967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39845e-8a8a-4511-ac83-015b413b52d3 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:49:59.707857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5c39845e-8a8a-4511-ac83-015b413b52d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:49:59.764240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39845e-8a8a-4511-ac83-015b413b52d3 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:49:59.769018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b4549-927e-442e-8103-495d97043abd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:49:59.769854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b4549-927e-442e-8103-495d97043abd service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:49:59.770246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b4549-927e-442e-8103-495d97043abd service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:49:59.770616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b4549-927e-442e-8103-495d97043abd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:49:59.780148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b4549-927e-442e-8103-495d97043abd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:49:59.782252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b4549-927e-442e-8103-495d97043abd service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:49:59.792546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 995/2018] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:49:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:49:59.795110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:00.110757 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bcb6f-65a4-48e6-9445-8b8dad72caa2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:00.120134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bcb6f-65a4-48e6-9445-8b8dad72caa2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:00.124079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb98e93-ffc9-4ba5-8878-9025b92287fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:00.124710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb98e93-ffc9-4ba5-8878-9025b92287fc service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:00.124996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb98e93-ffc9-4ba5-8878-9025b92287fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:00.125314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb98e93-ffc9-4ba5-8878-9025b92287fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:00.181437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb98e93-ffc9-4ba5-8878-9025b92287fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:00.182906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb98e93-ffc9-4ba5-8878-9025b92287fc service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:00.192501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1024/2019] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:00.678809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:01.313370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:01.840681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:01.878467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:02.707236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:02.741070 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:03.178167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:03.315167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:03.507189 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:03.764578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:03.800731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:04.632770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:04.710175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:05.776586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:06.062242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:07.033124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:07.332319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:08.133262 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48cdde-ba1a-4556-84fe-de0732bf362b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:08.136171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48cdde-ba1a-4556-84fe-de0732bf362b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:08.136171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48cdde-ba1a-4556-84fe-de0732bf362b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:08.154847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ed48cdde-ba1a-4556-84fe-de0732bf362b None None] MFA Rules not processed for user `0713ce790d0f46678e6402e290d511b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:08.156048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ed48cdde-ba1a-4556-84fe-de0732bf362b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:08.211289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 996/2020] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:08.556272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:08.769789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:09.525762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb98e93-ffc9-4ba5-8878-9025b92287fc service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:09.525762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8fb98e93-ffc9-4ba5-8878-9025b92287fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:09.587160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb98e93-ffc9-4ba5-8878-9025b92287fc service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:09.591753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:09.592385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:09.592670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:09.592947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] PATH_INFO: `/v3/users/4d1d51e9c1b44783b5c5931fa2e4ef72` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:09.594114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1d51e9c1b44783b5c5931fa2e4ef72)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:09.596716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:09.882867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1d51e9c1b44783b5c5931fa2e4ef72'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:09.898469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1d51e9c1b44783b5c5931fa2e4ef72'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:10.012238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:10.041258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1d51e9c1b44783b5c5931fa2e4ef72'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:10.042738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1d51e9c1b44783b5c5931fa2e4ef72'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:10.054588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1d51e9c1b44783b5c5931fa2e4ef72'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:10.061541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1025/2021] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:09 2022] DELETE /identity/v3/users/4d1d51e9c1b44783b5c5931fa2e4ef72 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:10.066228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31135a3a-d244-4af6-9fa2-73ee36621fad None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:10.066717 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31135a3a-d244-4af6-9fa2-73ee36621fad None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:10.066887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31135a3a-d244-4af6-9fa2-73ee36621fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:10.079957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:10.091790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-31135a3a-d244-4af6-9fa2-73ee36621fad None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:10.093084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-31135a3a-d244-4af6-9fa2-73ee36621fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:10.153260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 997/2022] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:10.163803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:10.165287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:10.239203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1a080-3c61-48e9-beec-18468ba5a1dc admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:10.243803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a89d2-7ee0-4d92-8810-c3966a618cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:10.244096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a89d2-7ee0-4d92-8810-c3966a618cbb service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:10.244276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a89d2-7ee0-4d92-8810-c3966a618cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:10.244528 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a89d2-7ee0-4d92-8810-c3966a618cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:10.254392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a89d2-7ee0-4d92-8810-c3966a618cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:10.256402 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a89d2-7ee0-4d92-8810-c3966a618cbb service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:10.266193 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1026/2023] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:10.772156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31135a3a-d244-4af6-9fa2-73ee36621fad None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:10.773638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-31135a3a-d244-4af6-9fa2-73ee36621fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:10.791937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31135a3a-d244-4af6-9fa2-73ee36621fad None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:10.796879 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:10.797570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:10.797836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:10.798309 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] PATH_INFO: `/v3/projects/38891ab9e3014b67b8f431ac9b4b1140` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:10.799754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38891ab9e3014b67b8f431ac9b4b1140)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:10.803854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:10.967874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38891ab9e3014b67b8f431ac9b4b1140', 'request_id': 'req-c7d7dba3-b570-4725-8ba7-fba4c4e49803'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:11.109168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:11.112762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38891ab9e3014b67b8f431ac9b4b1140', 'request_id': 'req-c7d7dba3-b570-4725-8ba7-fba4c4e49803'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:11.120868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38891ab9e3014b67b8f431ac9b4b1140', 'request_id': 'req-c7d7dba3-b570-4725-8ba7-fba4c4e49803'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:11.147685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] The token cache is being invalidate because project 38891ab9e3014b67b8f431ac9b4b1140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:11.147983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:11.148826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:11.150465 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 998/2024] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:10 2022] DELETE /identity/v3/projects/38891ab9e3014b67b8f431ac9b4b1140 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:11.155152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a89d2-7ee0-4d92-8810-c3966a618cbb service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:11.166030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a89d2-7ee0-4d92-8810-c3966a618cbb service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:11.255096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:11.255769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:11.256049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:11.256324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] PATH_INFO: `/v3/users/de8964b856be49eaadfe7a1e97ba3c16` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:11.258043 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8964b856be49eaadfe7a1e97ba3c16)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:11.259756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:11.492307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:11.517304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8964b856be49eaadfe7a1e97ba3c16'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:11.529870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8964b856be49eaadfe7a1e97ba3c16'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:11.675495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8964b856be49eaadfe7a1e97ba3c16'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:11.676108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8964b856be49eaadfe7a1e97ba3c16'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:11.692446 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a7c4d59f-0b2f-489f-af76-895af9b0e3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8964b856be49eaadfe7a1e97ba3c16'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:11.702917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:11.704572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:11.718232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1027/2025] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:11 2022] DELETE /identity/v3/users/de8964b856be49eaadfe7a1e97ba3c16 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:11.721948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7dba3-b570-4725-8ba7-fba4c4e49803 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:11.788382 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:11.789272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:11.789884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:11.790272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] PATH_INFO: `/v3/users/e318fcfe071c47e5a35165a10828b55d` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:11.791955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] RBAC: Authorizing `identity:delete_user(user_id=e318fcfe071c47e5a35165a10828b55d)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:11.803810 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:12.096461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e318fcfe071c47e5a35165a10828b55d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:12.217322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e318fcfe071c47e5a35165a10828b55d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:12.218142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e318fcfe071c47e5a35165a10828b55d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:12.228395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e318fcfe071c47e5a35165a10828b55d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:12.245092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e318fcfe071c47e5a35165a10828b55d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:12.256581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 999/2026] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:11 2022] DELETE /identity/v3/users/e318fcfe071c47e5a35165a10828b55d => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:12.261118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d25795-7c21-4706-8494-d9660fc33392 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:12.261592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d25795-7c21-4706-8494-d9660fc33392 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:12.261671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d25795-7c21-4706-8494-d9660fc33392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:12.294091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-72d25795-7c21-4706-8494-d9660fc33392 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:12.296102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-72d25795-7c21-4706-8494-d9660fc33392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:12.370148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1028/2027] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:50:12.394446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:12.396352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:12.463029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c7059-ce13-4f13-b5a8-12902c34b9be admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:12.466788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c9534-0aec-48fc-85c7-2af5efc671d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:12.467395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c9534-0aec-48fc-85c7-2af5efc671d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:12.467670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c9534-0aec-48fc-85c7-2af5efc671d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:12.467937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c9534-0aec-48fc-85c7-2af5efc671d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:12.480371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c9534-0aec-48fc-85c7-2af5efc671d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:12.482100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c9534-0aec-48fc-85c7-2af5efc671d2 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:12.494450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1000/2028] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:12.541390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d25795-7c21-4706-8494-d9660fc33392 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:12.543063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-72d25795-7c21-4706-8494-d9660fc33392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:12.556270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d25795-7c21-4706-8494-d9660fc33392 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:12.561367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:12.562345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:12.562910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:12.563340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] PATH_INFO: `/v3/projects/0202b7c6ba6348d68b74c188c3afa5ca` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:12.565071 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0202b7c6ba6348d68b74c188c3afa5ca)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:12.567442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:12.694158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0202b7c6ba6348d68b74c188c3afa5ca', 'request_id': 'req-73a93f34-fda0-4c53-a46d-78f4c61bae08'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:12.701162 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0202b7c6ba6348d68b74c188c3afa5ca', 'request_id': 'req-73a93f34-fda0-4c53-a46d-78f4c61bae08'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:12.814308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0202b7c6ba6348d68b74c188c3afa5ca', 'request_id': 'req-73a93f34-fda0-4c53-a46d-78f4c61bae08'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:12.855859 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] The token cache is being invalidate because project 0202b7c6ba6348d68b74c188c3afa5ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:12.857336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:12.858943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73a93f34-fda0-4c53-a46d-78f4c61bae08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:12.871286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b60a89-cb3d-465e-a32d-55b482a36b40 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:12.872003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b60a89-cb3d-465e-a32d-55b482a36b40 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:12.872420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b60a89-cb3d-465e-a32d-55b482a36b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:12.876784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1029/2029] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:12 2022] DELETE /identity/v3/projects/0202b7c6ba6348d68b74c188c3afa5ca => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:12.935170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-70b60a89-cb3d-465e-a32d-55b482a36b40 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:12.939594 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-70b60a89-cb3d-465e-a32d-55b482a36b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:13.147049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f700d6-ebd2-44fc-8124-1354164ea909 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:13.147373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f700d6-ebd2-44fc-8124-1354164ea909 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:13.154242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f700d6-ebd2-44fc-8124-1354164ea909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:13.154401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1001/2030] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:13.170119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a4f700d6-ebd2-44fc-8124-1354164ea909 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:13.172134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a4f700d6-ebd2-44fc-8124-1354164ea909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:13.189988 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1030/2031] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:13.194334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b60a89-cb3d-465e-a32d-55b482a36b40 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:13.211038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b60a89-cb3d-465e-a32d-55b482a36b40 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:13.216524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:13.219801 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b66418-b05c-469b-8060-1165bb4bd523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:13.221961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b66418-b05c-469b-8060-1165bb4bd523 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:13.222426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b66418-b05c-469b-8060-1165bb4bd523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:13.222848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b66418-b05c-469b-8060-1165bb4bd523 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:13.224063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b66418-b05c-469b-8060-1165bb4bd523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:50:13.224511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b66418-b05c-469b-8060-1165bb4bd523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:13.226476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b66418-b05c-469b-8060-1165bb4bd523 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:13.240843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1002/2032] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:50:13.249343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f700d6-ebd2-44fc-8124-1354164ea909 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:13.259189 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f700d6-ebd2-44fc-8124-1354164ea909 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:13.264956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216ba79-b781-40c6-957a-9f5802e1fde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:13.275048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216ba79-b781-40c6-957a-9f5802e1fde8 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:13.275390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216ba79-b781-40c6-957a-9f5802e1fde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:13.275675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216ba79-b781-40c6-957a-9f5802e1fde8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:13.294102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216ba79-b781-40c6-957a-9f5802e1fde8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:13.294102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216ba79-b781-40c6-957a-9f5802e1fde8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:13.477191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b66418-b05c-469b-8060-1165bb4bd523 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:13.477628 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1031/2033] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:13 2022] POST /identity/v3/projects => generated 383 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:13.490792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b66418-b05c-469b-8060-1165bb4bd523 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:13.519552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216ba79-b781-40c6-957a-9f5802e1fde8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:13.521279 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4216ba79-b781-40c6-957a-9f5802e1fde8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:13.580452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57db83a-9773-47cb-8d6d-2be05dc382ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:13.581494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57db83a-9773-47cb-8d6d-2be05dc382ba admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:13.581829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57db83a-9773-47cb-8d6d-2be05dc382ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:13.582225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57db83a-9773-47cb-8d6d-2be05dc382ba admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:13.583040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57db83a-9773-47cb-8d6d-2be05dc382ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:13.585215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57db83a-9773-47cb-8d6d-2be05dc382ba admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:13.673010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1003/2034] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:13 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:13.677815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57db83a-9773-47cb-8d6d-2be05dc382ba admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:13.702673 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216ba79-b781-40c6-957a-9f5802e1fde8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:13.704238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57db83a-9773-47cb-8d6d-2be05dc382ba admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:13.716317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89961844-c7c5-4679-a1fa-86410a9c95c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:13.716950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961844-c7c5-4679-a1fa-86410a9c95c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:13.717228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961844-c7c5-4679-a1fa-86410a9c95c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:13.717470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:13.718523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961844-c7c5-4679-a1fa-86410a9c95c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:13.719171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961844-c7c5-4679-a1fa-86410a9c95c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:13.719599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:13.720004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:13.720288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] PATH_INFO: `/v3/projects/1a573da49b7f4c799e51cc4115314f35` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:13.723108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961844-c7c5-4679-a1fa-86410a9c95c6 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:13.725648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a573da49b7f4c799e51cc4115314f35)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:13.728651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:13.752644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1004/2035] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:13.755453 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89961844-c7c5-4679-a1fa-86410a9c95c6 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:13.779386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89961844-c7c5-4679-a1fa-86410a9c95c6 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:13.792892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3ba2a-e461-4a24-835f-f097646f6c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:13.793521 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3ba2a-e461-4a24-835f-f097646f6c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:13.794229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3ba2a-e461-4a24-835f-f097646f6c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:13.794736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3ba2a-e461-4a24-835f-f097646f6c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:13.795395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3ba2a-e461-4a24-835f-f097646f6c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:13.797011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3ba2a-e461-4a24-835f-f097646f6c08 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:13.815821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1005/2036] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:13.820070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3ba2a-e461-4a24-835f-f097646f6c08 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:13.832592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3ba2a-e461-4a24-835f-f097646f6c08 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:13.838691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36faa66d-c983-47c3-b025-e9867c8236de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:13.851811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faa66d-c983-47c3-b025-e9867c8236de admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:13.851811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faa66d-c983-47c3-b025-e9867c8236de admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:13.851811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faa66d-c983-47c3-b025-e9867c8236de admin admin] PATH_INFO: `/v3/projects/c6095557b0e4424688f718be45b99b31/users/5fb0f582c1874dda9749c5197eca2ca6/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:13.866312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36faa66d-c983-47c3-b025-e9867c8236de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6095557b0e4424688f718be45b99b31, user_id=5fb0f582c1874dda9749c5197eca2ca6, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:13.869283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36faa66d-c983-47c3-b025-e9867c8236de admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:13.987255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1006/2037] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:13 2022] PUT /identity/v3/projects/c6095557b0e4424688f718be45b99b31/users/5fb0f582c1874dda9749c5197eca2ca6/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:13.991606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8b9b-5ed3-44b5-97eb-4da8d5c2301e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:13.991606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8b9b-5ed3-44b5-97eb-4da8d5c2301e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:13.991606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8b9b-5ed3-44b5-97eb-4da8d5c2301e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:13.999010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a573da49b7f4c799e51cc4115314f35', 'request_id': 'req-fa91a480-a6c5-4720-8239-c5d9d95053ad'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:14.023945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:14.037878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a573da49b7f4c799e51cc4115314f35', 'request_id': 'req-fa91a480-a6c5-4720-8239-c5d9d95053ad'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:14.053544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b50d8b9b-5ed3-44b5-97eb-4da8d5c2301e None None] MFA Rules not processed for user `5fb0f582c1874dda9749c5197eca2ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:14.055169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b50d8b9b-5ed3-44b5-97eb-4da8d5c2301e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:14.132001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1007/2038] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:14.136714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d8b9b-5ed3-44b5-97eb-4da8d5c2301e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:14.166218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a573da49b7f4c799e51cc4115314f35', 'request_id': 'req-fa91a480-a6c5-4720-8239-c5d9d95053ad'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:14.180862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] The token cache is being invalidate because project 1a573da49b7f4c799e51cc4115314f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:14.181302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:14.182102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:14.184543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1032/2039] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:13 2022] DELETE /identity/v3/projects/1a573da49b7f4c799e51cc4115314f35 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:14.190231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:14.191862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:14.227436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d8b9b-5ed3-44b5-97eb-4da8d5c2301e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:14.233586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41609a8-ae01-4304-b701-20c89e3016e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:14.235269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41609a8-ae01-4304-b701-20c89e3016e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:14.237219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41609a8-ae01-4304-b701-20c89e3016e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:14.237680 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41609a8-ae01-4304-b701-20c89e3016e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:14.243303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41609a8-ae01-4304-b701-20c89e3016e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:14.243303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41609a8-ae01-4304-b701-20c89e3016e8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:14.277131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91a480-a6c5-4720-8239-c5d9d95053ad admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:14.284055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:14.295706 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:14.297908 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:14.298343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] PATH_INFO: `/v3/users/2ce7a9bfd8ed46f2a0d370e2dbff7c8b` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:14.302467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce7a9bfd8ed46f2a0d370e2dbff7c8b)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:14.307765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:14.375763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1008/2040] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:14 2022] POST /identity/v3/projects => generated 383 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:14.379712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41609a8-ae01-4304-b701-20c89e3016e8 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:14.382387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f41609a8-ae01-4304-b701-20c89e3016e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:14.503441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41609a8-ae01-4304-b701-20c89e3016e8 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:14.508316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef7774-e650-465b-a8a9-2273e28d08d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:14.508933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef7774-e650-465b-a8a9-2273e28d08d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:14.509216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef7774-e650-465b-a8a9-2273e28d08d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:14.509487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef7774-e650-465b-a8a9-2273e28d08d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:14.510221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef7774-e650-465b-a8a9-2273e28d08d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:14.512157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef7774-e650-465b-a8a9-2273e28d08d0 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:14.592788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce7a9bfd8ed46f2a0d370e2dbff7c8b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:14.621111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1009/2041] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:14 2022] POST /identity/v3/users => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:14.622747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce7a9bfd8ed46f2a0d370e2dbff7c8b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:14.624731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef7774-e650-465b-a8a9-2273e28d08d0 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:14.639587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef7774-e650-465b-a8a9-2273e28d08d0 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:14.645308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0dd8f-79d2-44cc-b95c-9fa6bec82ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:14.647843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0dd8f-79d2-44cc-b95c-9fa6bec82ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:14.647843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0dd8f-79d2-44cc-b95c-9fa6bec82ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:14.647843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0dd8f-79d2-44cc-b95c-9fa6bec82ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:14.647843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0dd8f-79d2-44cc-b95c-9fa6bec82ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:14.648725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0dd8f-79d2-44cc-b95c-9fa6bec82ea0 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:14.659496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1010/2042] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:50:14.663531 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0dd8f-79d2-44cc-b95c-9fa6bec82ea0 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:14.677976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0dd8f-79d2-44cc-b95c-9fa6bec82ea0 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:14.684097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509cafe-ad29-4854-b7af-30c31042e5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:14.684862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509cafe-ad29-4854-b7af-30c31042e5b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:14.685239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509cafe-ad29-4854-b7af-30c31042e5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:14.686586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509cafe-ad29-4854-b7af-30c31042e5b4 admin admin] PATH_INFO: `/v3/projects/9348eba198dd4d68ba069cc45ae1366c/users/0a302855362c4907b9720541428c05e7/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:14.701233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce7a9bfd8ed46f2a0d370e2dbff7c8b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:14.702032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce7a9bfd8ed46f2a0d370e2dbff7c8b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:14.712268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce7a9bfd8ed46f2a0d370e2dbff7c8b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:14.730236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509cafe-ad29-4854-b7af-30c31042e5b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9348eba198dd4d68ba069cc45ae1366c, user_id=0a302855362c4907b9720541428c05e7, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:14.731420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1033/2043] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:14 2022] DELETE /identity/v3/users/2ce7a9bfd8ed46f2a0d370e2dbff7c8b => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:14.743147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509cafe-ad29-4854-b7af-30c31042e5b4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:14.906887 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1011/2044] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:14 2022] PUT /identity/v3/projects/9348eba198dd4d68ba069cc45ae1366c/users/0a302855362c4907b9720541428c05e7/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:14.910751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:14.913022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:15.021174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b167c-a977-4a13-99aa-a1ccb603ce18 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:15.112924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22192ba-a7bd-4cf1-9c7b-324d998d3dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:15.113562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22192ba-a7bd-4cf1-9c7b-324d998d3dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:15.113975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22192ba-a7bd-4cf1-9c7b-324d998d3dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:15.114160 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22192ba-a7bd-4cf1-9c7b-324d998d3dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:15.114824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22192ba-a7bd-4cf1-9c7b-324d998d3dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:15.117203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22192ba-a7bd-4cf1-9c7b-324d998d3dc8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:15.129162 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1034/2045] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:15.133194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509cafe-ad29-4854-b7af-30c31042e5b4 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:15.150859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509cafe-ad29-4854-b7af-30c31042e5b4 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:15.159245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f72af3-a6a1-4b1d-ba81-c207985ec6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:15.163170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72af3-a6a1-4b1d-ba81-c207985ec6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:15.165163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72af3-a6a1-4b1d-ba81-c207985ec6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:15.165555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72af3-a6a1-4b1d-ba81-c207985ec6e7 admin admin] PATH_INFO: `/v3/projects/9348eba198dd4d68ba069cc45ae1366c/users/0a302855362c4907b9720541428c05e7/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:15.168791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f72af3-a6a1-4b1d-ba81-c207985ec6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9348eba198dd4d68ba069cc45ae1366c, user_id=0a302855362c4907b9720541428c05e7, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:15.178093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f72af3-a6a1-4b1d-ba81-c207985ec6e7 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:15.223533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1012/2046] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:15 2022] PUT /identity/v3/projects/9348eba198dd4d68ba069cc45ae1366c/users/0a302855362c4907b9720541428c05e7/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:15.234613 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd57f81-6900-46f8-a5e1-4bf26940635e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:15.235020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd57f81-6900-46f8-a5e1-4bf26940635e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:15.235409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd57f81-6900-46f8-a5e1-4bf26940635e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:15.298213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3cd57f81-6900-46f8-a5e1-4bf26940635e None None] MFA Rules not processed for user `0a302855362c4907b9720541428c05e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:15.300078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3cd57f81-6900-46f8-a5e1-4bf26940635e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:15.409518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1035/2047] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:15.426983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f72af3-a6a1-4b1d-ba81-c207985ec6e7 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:15.428612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-60f72af3-a6a1-4b1d-ba81-c207985ec6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:15.483489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd57f81-6900-46f8-a5e1-4bf26940635e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:15.485020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3cd57f81-6900-46f8-a5e1-4bf26940635e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:15.503426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f72af3-a6a1-4b1d-ba81-c207985ec6e7 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:15.507786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f4622-1667-45dc-abeb-6bcf66600e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:15.508552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f4622-1667-45dc-abeb-6bcf66600e6d service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:15.509090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f4622-1667-45dc-abeb-6bcf66600e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:15.509520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f4622-1667-45dc-abeb-6bcf66600e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:15.524477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f4622-1667-45dc-abeb-6bcf66600e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:15.526025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f4622-1667-45dc-abeb-6bcf66600e6d service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:15.549678 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1013/2048] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:15.595464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd57f81-6900-46f8-a5e1-4bf26940635e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:15.600968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:15.601885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:15.602237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:15.602583 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] PATH_INFO: `/v3/projects/2e26e3c8ed40450489075a614b0d4465` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:15.603939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e26e3c8ed40450489075a614b0d4465)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:15.606861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:15.718539 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:15.745034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f4622-1667-45dc-abeb-6bcf66600e6d service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:15.754630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f4622-1667-45dc-abeb-6bcf66600e6d service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:15.774785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd4608-e4f8-4f9f-ae62-313544861d99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:15.775410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4608-e4f8-4f9f-ae62-313544861d99 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:15.775883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4608-e4f8-4f9f-ae62-313544861d99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:15.781626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4608-e4f8-4f9f-ae62-313544861d99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:15.781626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-cecd4608-e4f8-4f9f-ae62-313544861d99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:15.845229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e26e3c8ed40450489075a614b0d4465', 'request_id': 'req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:15.859521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e26e3c8ed40450489075a614b0d4465', 'request_id': 'req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:15.881277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd4608-e4f8-4f9f-ae62-313544861d99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:15.883661 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd4608-e4f8-4f9f-ae62-313544861d99 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:15.911146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1014/2049] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:50:15.972316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:16.009567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e26e3c8ed40450489075a614b0d4465', 'request_id': 'req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:16.028806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] The token cache is being invalidate because project 2e26e3c8ed40450489075a614b0d4465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:16.029377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:16.030682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d8e5cf68-a8eb-4318-afc0-e7e91de633cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:16.033748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1036/2050] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:15 2022] DELETE /identity/v3/projects/2e26e3c8ed40450489075a614b0d4465 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:16.040438 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e751311-169c-4ac4-8d9c-82bb4893b1dc None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.041970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e751311-169c-4ac4-8d9c-82bb4893b1dc None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.042708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e751311-169c-4ac4-8d9c-82bb4893b1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.106653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-5e751311-169c-4ac4-8d9c-82bb4893b1dc None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:16.106653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5e751311-169c-4ac4-8d9c-82bb4893b1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:16.236134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1015/2051] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:16.243986 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1ef05-abec-4771-95e0-bc457f120920 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.244273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1ef05-abec-4771-95e0-bc457f120920 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.249609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1ef05-abec-4771-95e0-bc457f120920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.266621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f9f1ef05-abec-4771-95e0-bc457f120920 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:16.267967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f9f1ef05-abec-4771-95e0-bc457f120920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:16.282212 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1037/2052] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:16.283457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e751311-169c-4ac4-8d9c-82bb4893b1dc None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:16.298306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e751311-169c-4ac4-8d9c-82bb4893b1dc None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:16.306014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec83d7f-d295-435e-90ab-f8167d592850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:16.306716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83d7f-d295-435e-90ab-f8167d592850 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.307281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83d7f-d295-435e-90ab-f8167d592850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.307630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83d7f-d295-435e-90ab-f8167d592850 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.308352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec83d7f-d295-435e-90ab-f8167d592850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:50:16.308723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec83d7f-d295-435e-90ab-f8167d592850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:16.310599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec83d7f-d295-435e-90ab-f8167d592850 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:16.334960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1016/2053] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:50:16.336013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1ef05-abec-4771-95e0-bc457f120920 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:16.352557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1ef05-abec-4771-95e0-bc457f120920 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:16.367773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86a8ef-f042-4962-ba67-e77c2927e4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:16.370806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86a8ef-f042-4962-ba67-e77c2927e4ae admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.371849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86a8ef-f042-4962-ba67-e77c2927e4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.372794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86a8ef-f042-4962-ba67-e77c2927e4ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.374704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86a8ef-f042-4962-ba67-e77c2927e4ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:16.377887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86a8ef-f042-4962-ba67-e77c2927e4ae admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:16.539660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1038/2054] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:16 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:16.544028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec83d7f-d295-435e-90ab-f8167d592850 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:16.554802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec83d7f-d295-435e-90ab-f8167d592850 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:16.608756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86a8ef-f042-4962-ba67-e77c2927e4ae admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:16.610781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5c86a8ef-f042-4962-ba67-e77c2927e4ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:16.622788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583db0e-1da6-487f-aaba-00ad774bda1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:16.623340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583db0e-1da6-487f-aaba-00ad774bda1c admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.623617 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583db0e-1da6-487f-aaba-00ad774bda1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.623892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583db0e-1da6-487f-aaba-00ad774bda1c admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.624561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583db0e-1da6-487f-aaba-00ad774bda1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:16.626524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583db0e-1da6-487f-aaba-00ad774bda1c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:16.678439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86a8ef-f042-4962-ba67-e77c2927e4ae admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:16.683483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:16.684123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.684407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.684691 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] PATH_INFO: `/v3/users/5fb0f582c1874dda9749c5197eca2ca6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.685899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb0f582c1874dda9749c5197eca2ca6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:16.687856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:16.758147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1017/2055] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:16 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:16.758147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d29c19-3314-4cd7-a8d1-7cd0f89ff400 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.758147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d29c19-3314-4cd7-a8d1-7cd0f89ff400 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.758147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d29c19-3314-4cd7-a8d1-7cd0f89ff400 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.758726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1018/2056] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:50:16.759345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d29c19-3314-4cd7-a8d1-7cd0f89ff400 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:16.772301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d29c19-3314-4cd7-a8d1-7cd0f89ff400 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:16.777706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a7f75-2d60-4b2d-a6f7-39bc17a60507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:16.778317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a7f75-2d60-4b2d-a6f7-39bc17a60507 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.779061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a7f75-2d60-4b2d-a6f7-39bc17a60507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.779061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a7f75-2d60-4b2d-a6f7-39bc17a60507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.779602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a7f75-2d60-4b2d-a6f7-39bc17a60507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:16.781224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a7f75-2d60-4b2d-a6f7-39bc17a60507 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:16.793568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1019/2057] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:16.795133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21ac8-de11-4cb0-beca-c320d9eb9d79 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.795383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21ac8-de11-4cb0-beca-c320d9eb9d79 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.795663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21ac8-de11-4cb0-beca-c320d9eb9d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.817226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-bcd21ac8-de11-4cb0-beca-c320d9eb9d79 None None] MFA Rules not processed for user `34ec5c9c71d14762a4869b7d66e87ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:16.818441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bcd21ac8-de11-4cb0-beca-c320d9eb9d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:16.895458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1020/2058] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:50:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:16.896635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd21ac8-de11-4cb0-beca-c320d9eb9d79 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:16.919768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd21ac8-de11-4cb0-beca-c320d9eb9d79 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:16.926233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745289a-a4df-4f3a-a306-951c187afe41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:16.927092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745289a-a4df-4f3a-a306-951c187afe41 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.927667 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745289a-a4df-4f3a-a306-951c187afe41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.928365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745289a-a4df-4f3a-a306-951c187afe41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.929231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745289a-a4df-4f3a-a306-951c187afe41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:16.931781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745289a-a4df-4f3a-a306-951c187afe41 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:16.936431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb0f582c1874dda9749c5197eca2ca6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:16.949750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1021/2059] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:16.949750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745289a-a4df-4f3a-a306-951c187afe41 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:16.959424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f745289a-a4df-4f3a-a306-951c187afe41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:16.960654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb0f582c1874dda9749c5197eca2ca6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:16.974867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745289a-a4df-4f3a-a306-951c187afe41 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:16.980918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16d97-3ded-4a80-989d-92a631bc028d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ec5c9c71d14762a4869b7d66e87ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:16.981421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16d97-3ded-4a80-989d-92a631bc028d service placement] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:16.981705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16d97-3ded-4a80-989d-92a631bc028d service placement] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:16.982014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16d97-3ded-4a80-989d-92a631bc028d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:16.990664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d16d97-3ded-4a80-989d-92a631bc028d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:16.992252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d16d97-3ded-4a80-989d-92a631bc028d service placement] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:17.002692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1022/2060] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:17.003514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16d97-3ded-4a80-989d-92a631bc028d service placement] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:17.005077 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-72d16d97-3ded-4a80-989d-92a631bc028d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:17.029835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16d97-3ded-4a80-989d-92a631bc028d service placement] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:17.036649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:17.037252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:17.037534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:17.037841 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] PATH_INFO: `/v3/users/c182a0a100054e7681d27d3439aa03f1` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:17.039694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c182a0a100054e7681d27d3439aa03f1)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:17.041781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:17.174991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb0f582c1874dda9749c5197eca2ca6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:17.175651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb0f582c1874dda9749c5197eca2ca6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:17.183348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb0f582c1874dda9749c5197eca2ca6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:17.190052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1039/2061] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:16 2022] DELETE /identity/v3/users/5fb0f582c1874dda9749c5197eca2ca6 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:17.190982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:17.192433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:17.264654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491cf50-c3f9-4c11-becd-1e5bee1ad31e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:17.271605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940de65b-1415-47d2-b3bc-013875a92c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:17.271605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940de65b-1415-47d2-b3bc-013875a92c9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:17.271605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940de65b-1415-47d2-b3bc-013875a92c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:17.271605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940de65b-1415-47d2-b3bc-013875a92c9b admin admin] PATH_INFO: `/v3/projects/463a97b3425a4ada9253dce1047b29e8/users/b85f61e5aa5140059d88dc49996e57ed/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:17.287636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940de65b-1415-47d2-b3bc-013875a92c9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=463a97b3425a4ada9253dce1047b29e8, user_id=b85f61e5aa5140059d88dc49996e57ed, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:17.290630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940de65b-1415-47d2-b3bc-013875a92c9b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:17.447236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c182a0a100054e7681d27d3439aa03f1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:17.465916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1040/2062] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:17 2022] PUT /identity/v3/projects/463a97b3425a4ada9253dce1047b29e8/users/b85f61e5aa5140059d88dc49996e57ed/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:17.468031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c96cf6-dc53-4235-ab71-9532acbf6fbc None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:17.468323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c96cf6-dc53-4235-ab71-9532acbf6fbc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:17.468601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c96cf6-dc53-4235-ab71-9532acbf6fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:17.514702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e7c96cf6-dc53-4235-ab71-9532acbf6fbc None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:17.515954 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e7c96cf6-dc53-4235-ab71-9532acbf6fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:17.542793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c182a0a100054e7681d27d3439aa03f1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:17.543593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c182a0a100054e7681d27d3439aa03f1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:17.553080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c182a0a100054e7681d27d3439aa03f1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:17.567901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0dd93262-9401-4c67-b5d5-a57980e68d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c182a0a100054e7681d27d3439aa03f1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:17.578261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1023/2063] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:17 2022] DELETE /identity/v3/users/c182a0a100054e7681d27d3439aa03f1 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:17.578261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f623fd8-1d04-4daa-9e0a-180cdbf6403c None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:17.578261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f623fd8-1d04-4daa-9e0a-180cdbf6403c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:17.578261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f623fd8-1d04-4daa-9e0a-180cdbf6403c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:17.627555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4f623fd8-1d04-4daa-9e0a-180cdbf6403c None None] MFA Rules not processed for user `b85f61e5aa5140059d88dc49996e57ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:17.628930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4f623fd8-1d04-4daa-9e0a-180cdbf6403c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:17.629195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1041/2064] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:17.653347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c96cf6-dc53-4235-ab71-9532acbf6fbc None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:17.655383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e7c96cf6-dc53-4235-ab71-9532acbf6fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:17.728182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1024/2065] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:17.734193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f623fd8-1d04-4daa-9e0a-180cdbf6403c None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:17.737320 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4f623fd8-1d04-4daa-9e0a-180cdbf6403c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:17.783907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c96cf6-dc53-4235-ab71-9532acbf6fbc None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:17.841580 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f623fd8-1d04-4daa-9e0a-180cdbf6403c None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:17.847564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e65f6f-6d1d-4154-a3e4-21e08cd17473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:17.848249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e65f6f-6d1d-4154-a3e4-21e08cd17473 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:17.848532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e65f6f-6d1d-4154-a3e4-21e08cd17473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:17.848802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e65f6f-6d1d-4154-a3e4-21e08cd17473 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:17.849440 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e65f6f-6d1d-4154-a3e4-21e08cd17473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:17.853178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e65f6f-6d1d-4154-a3e4-21e08cd17473 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:17.875332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3110a-9054-4851-b588-7242522d908a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:17.876271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3110a-9054-4851-b588-7242522d908a service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:17.877989 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3110a-9054-4851-b588-7242522d908a service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:17.878289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3110a-9054-4851-b588-7242522d908a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:17.891505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3110a-9054-4851-b588-7242522d908a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:17.893441 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3110a-9054-4851-b588-7242522d908a service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:17.903113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1042/2066] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:17.960613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1025/2067] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:17 2022] POST /identity/v3/projects => generated 383 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:17.965248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3110a-9054-4851-b588-7242522d908a service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:17.976031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3110a-9054-4851-b588-7242522d908a service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:17.994716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e65f6f-6d1d-4154-a3e4-21e08cd17473 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:17.996420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d4e65f6f-6d1d-4154-a3e4-21e08cd17473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:18.076612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6650274-f077-4283-b8f6-d6820fe3b617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:18.077238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6650274-f077-4283-b8f6-d6820fe3b617 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.077523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6650274-f077-4283-b8f6-d6820fe3b617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.077836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6650274-f077-4283-b8f6-d6820fe3b617 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.078491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6650274-f077-4283-b8f6-d6820fe3b617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:18.080506 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6650274-f077-4283-b8f6-d6820fe3b617 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:18.085123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e65f6f-6d1d-4154-a3e4-21e08cd17473 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:18.090861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:18.091729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.092255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.092699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] PATH_INFO: `/v3/projects/5b4fba22f55549188446d54f94aa1620` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.094075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b4fba22f55549188446d54f94aa1620)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:18.096925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:18.244031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1043/2068] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:17 2022] POST /identity/v3/users => generated 298 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:18.248073 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6650274-f077-4283-b8f6-d6820fe3b617 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:18.258093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6650274-f077-4283-b8f6-d6820fe3b617 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:18.263286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bcca9-5790-4340-8f35-e68667c3001c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:18.264021 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bcca9-5790-4340-8f35-e68667c3001c admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.264274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bcca9-5790-4340-8f35-e68667c3001c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.264791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bcca9-5790-4340-8f35-e68667c3001c admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.265452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bcca9-5790-4340-8f35-e68667c3001c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:18.267095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bcca9-5790-4340-8f35-e68667c3001c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:18.278391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1044/2069] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:50:18.282443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bcca9-5790-4340-8f35-e68667c3001c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:18.292830 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bcca9-5790-4340-8f35-e68667c3001c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:18.297500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a107e1-2155-49fa-8946-b65fea2cc751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:18.298184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a107e1-2155-49fa-8946-b65fea2cc751 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.298488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a107e1-2155-49fa-8946-b65fea2cc751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.298749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a107e1-2155-49fa-8946-b65fea2cc751 admin admin] PATH_INFO: `/v3/projects/24b8780568c14aeb8acdfe076c6fbefb/users/24e73cdc80594548bd1002ca28be8061/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.315405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a107e1-2155-49fa-8946-b65fea2cc751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b8780568c14aeb8acdfe076c6fbefb, user_id=24e73cdc80594548bd1002ca28be8061, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:18.319142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a107e1-2155-49fa-8946-b65fea2cc751 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:18.326146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b4fba22f55549188446d54f94aa1620', 'request_id': 'req-8c81b234-5ff4-44cb-96e9-b369a993b693'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:18.383458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:18.440366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b4fba22f55549188446d54f94aa1620', 'request_id': 'req-8c81b234-5ff4-44cb-96e9-b369a993b693'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:18.451250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b4fba22f55549188446d54f94aa1620', 'request_id': 'req-8c81b234-5ff4-44cb-96e9-b369a993b693'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:18.457688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1045/2070] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:18 2022] PUT /identity/v3/projects/24b8780568c14aeb8acdfe076c6fbefb/users/24e73cdc80594548bd1002ca28be8061/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:18.473265 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a107e1-2155-49fa-8946-b65fea2cc751 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:18.473907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] The token cache is being invalidate because project 5b4fba22f55549188446d54f94aa1620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:18.474029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:18.474871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:18.476130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1026/2071] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:17 2022] DELETE /identity/v3/projects/5b4fba22f55549188446d54f94aa1620 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:18.477042 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:18.480278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f0a107e1-2155-49fa-8946-b65fea2cc751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:18.480488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:18.552462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81b234-5ff4-44cb-96e9-b369a993b693 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:18.565761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a107e1-2155-49fa-8946-b65fea2cc751 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:18.570669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:18.571304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.574330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.574330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] PATH_INFO: `/v3/projects/c6095557b0e4424688f718be45b99b31` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.574330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6095557b0e4424688f718be45b99b31)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:18.575245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:18.598281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:18.616116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5c921-727b-44a6-ad45-b998ee0b81b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:18.616659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5c921-727b-44a6-ad45-b998ee0b81b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.616938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5c921-727b-44a6-ad45-b998ee0b81b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.617212 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5c921-727b-44a6-ad45-b998ee0b81b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.617889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5c921-727b-44a6-ad45-b998ee0b81b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:18.619509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5c921-727b-44a6-ad45-b998ee0b81b8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:18.630646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1027/2072] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:18.632087 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9a9e7-82f4-4685-974a-b73755d762dd None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.632555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9a9e7-82f4-4685-974a-b73755d762dd None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.632763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9a9e7-82f4-4685-974a-b73755d762dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.664516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-47f9a9e7-82f4-4685-974a-b73755d762dd None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:18.665767 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-47f9a9e7-82f4-4685-974a-b73755d762dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:18.674456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1028/2073] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:18.675159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9a9e7-82f4-4685-974a-b73755d762dd None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:18.684844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9a9e7-82f4-4685-974a-b73755d762dd None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:18.690354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db233f6-2ff6-40a9-b974-0b5e2e66de75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:18.691004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db233f6-2ff6-40a9-b974-0b5e2e66de75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.691281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db233f6-2ff6-40a9-b974-0b5e2e66de75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.691551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db233f6-2ff6-40a9-b974-0b5e2e66de75 admin admin] PATH_INFO: `/v3/projects/24b8780568c14aeb8acdfe076c6fbefb/users/24e73cdc80594548bd1002ca28be8061/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.693562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db233f6-2ff6-40a9-b974-0b5e2e66de75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b8780568c14aeb8acdfe076c6fbefb, user_id=24e73cdc80594548bd1002ca28be8061, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:18.696245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db233f6-2ff6-40a9-b974-0b5e2e66de75 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:18.714347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1029/2074] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:18 2022] PUT /identity/v3/projects/24b8780568c14aeb8acdfe076c6fbefb/users/24e73cdc80594548bd1002ca28be8061/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:18.714347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a403e-f1d6-496a-a7c5-b90d9d9d8bf2 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.714347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a403e-f1d6-496a-a7c5-b90d9d9d8bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.714347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a403e-f1d6-496a-a7c5-b90d9d9d8bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.727680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6095557b0e4424688f718be45b99b31', 'request_id': 'req-8125d5f6-b406-4770-999e-276fe11ae005'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:18.731751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-371a403e-f1d6-496a-a7c5-b90d9d9d8bf2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:18.732975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-371a403e-f1d6-496a-a7c5-b90d9d9d8bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:18.735038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6095557b0e4424688f718be45b99b31', 'request_id': 'req-8125d5f6-b406-4770-999e-276fe11ae005'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:18.805550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1030/2075] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:18.814768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502eb4-81d8-42d8-a93c-f39fcaba3234 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.814768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502eb4-81d8-42d8-a93c-f39fcaba3234 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.814768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502eb4-81d8-42d8-a93c-f39fcaba3234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.842149 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-46502eb4-81d8-42d8-a93c-f39fcaba3234 None None] MFA Rules not processed for user `24e73cdc80594548bd1002ca28be8061`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:18.843427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-46502eb4-81d8-42d8-a93c-f39fcaba3234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:18.869164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6095557b0e4424688f718be45b99b31', 'request_id': 'req-8125d5f6-b406-4770-999e-276fe11ae005'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:18.885646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] The token cache is being invalidate because project c6095557b0e4424688f718be45b99b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:18.885984 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:18.886681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:18.888744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1046/2076] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:18 2022] DELETE /identity/v3/projects/c6095557b0e4424688f718be45b99b31 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:18.889527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:18.891243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:18.906090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125d5f6-b406-4770-999e-276fe11ae005 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:18.912434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ea5a3-b301-4729-bf1a-3383d0311f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:18.913137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ea5a3-b301-4729-bf1a-3383d0311f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.914131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ea5a3-b301-4729-bf1a-3383d0311f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.914131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ea5a3-b301-4729-bf1a-3383d0311f5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.914367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ea5a3-b301-4729-bf1a-3383d0311f5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:50:18.914642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ea5a3-b301-4729-bf1a-3383d0311f5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:18.916427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ea5a3-b301-4729-bf1a-3383d0311f5e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:18.936683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1047/2077] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:50:18.938002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ea5a3-b301-4729-bf1a-3383d0311f5e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:18.940204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1e6ea5a3-b301-4729-bf1a-3383d0311f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:18.943503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1031/2078] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:18.948712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46502eb4-81d8-42d8-a93c-f39fcaba3234 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:18.951259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-46502eb4-81d8-42d8-a93c-f39fcaba3234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:18.967439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ea5a3-b301-4729-bf1a-3383d0311f5e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:18.967809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46502eb4-81d8-42d8-a93c-f39fcaba3234 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:18.973981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:18.974624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.974905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.975019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:18.975177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] PATH_INFO: `/v3/users/81a034ed67914a038386462cc6424314` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.975787 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:18.976234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:18.976347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81a034ed67914a038386462cc6424314)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:18.976673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] PATH_INFO: `/v3/users/0a302855362c4907b9720541428c05e7` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:18.978595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:18.978715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a302855362c4907b9720541428c05e7)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:18.981603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:19.185689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81a034ed67914a038386462cc6424314'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.198528 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81a034ed67914a038386462cc6424314'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.210094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a302855362c4907b9720541428c05e7'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.294672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81a034ed67914a038386462cc6424314'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.295221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81a034ed67914a038386462cc6424314'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.302999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81a034ed67914a038386462cc6424314'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.308843 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1048/2079] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:18 2022] DELETE /identity/v3/users/81a034ed67914a038386462cc6424314 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:19.309590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:19.310952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:19.354523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a302855362c4907b9720541428c05e7'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.355273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a302855362c4907b9720541428c05e7'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.379213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a302855362c4907b9720541428c05e7'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.388673 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796b841-2f7c-4f62-933d-a1e510153399 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:19.393462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a302855362c4907b9720541428c05e7'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.394834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe0a31-b6e5-4741-810b-4d5041235acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:19.395840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe0a31-b6e5-4741-810b-4d5041235acb admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:19.396321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe0a31-b6e5-4741-810b-4d5041235acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:19.396818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe0a31-b6e5-4741-810b-4d5041235acb admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:19.397766 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe0a31-b6e5-4741-810b-4d5041235acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:19.400947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe0a31-b6e5-4741-810b-4d5041235acb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:19.401716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1032/2080] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:18 2022] DELETE /identity/v3/users/0a302855362c4907b9720541428c05e7 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:19.405543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:19.407647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:19.478610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e56b0-fd56-441b-93eb-5b1755792be5 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:19.482362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10358a-58a0-49cf-b498-a1d7c65d2d02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:19.483039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10358a-58a0-49cf-b498-a1d7c65d2d02 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:19.483318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10358a-58a0-49cf-b498-a1d7c65d2d02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:19.483590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10358a-58a0-49cf-b498-a1d7c65d2d02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:19.489417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1d10358a-58a0-49cf-b498-a1d7c65d2d02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:19.516163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1049/2081] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:19 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:19.521637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe0a31-b6e5-4741-810b-4d5041235acb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:19.523531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c0fe0a31-b6e5-4741-810b-4d5041235acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:19.557999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10358a-58a0-49cf-b498-a1d7c65d2d02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:19.557999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10358a-58a0-49cf-b498-a1d7c65d2d02 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:19.566924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1033/2082] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:19.606013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe0a31-b6e5-4741-810b-4d5041235acb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:19.610715 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e5aa5-96d6-4b5a-b723-757e1202599b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:19.612668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e5aa5-96d6-4b5a-b723-757e1202599b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:19.612936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e5aa5-96d6-4b5a-b723-757e1202599b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:19.613284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e5aa5-96d6-4b5a-b723-757e1202599b admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:19.613918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e5aa5-96d6-4b5a-b723-757e1202599b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:19.615838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e5aa5-96d6-4b5a-b723-757e1202599b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:19.623155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10358a-58a0-49cf-b498-a1d7c65d2d02 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:19.627424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1d10358a-58a0-49cf-b498-a1d7c65d2d02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:19.722536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10358a-58a0-49cf-b498-a1d7c65d2d02 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:19.729358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:19.730363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:19.730865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:19.731749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] PATH_INFO: `/v3/projects/051134557f784863b566fb376a71ef49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:19.733061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=051134557f784863b566fb376a71ef49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:19.736623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:19.742104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1050/2083] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:19 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:19.748531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e5aa5-96d6-4b5a-b723-757e1202599b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:19.760320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e5aa5-96d6-4b5a-b723-757e1202599b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:19.765499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4cd6d-6842-4a2f-a100-d8deaa673950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:19.766227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4cd6d-6842-4a2f-a100-d8deaa673950 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:19.766699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4cd6d-6842-4a2f-a100-d8deaa673950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:19.767035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4cd6d-6842-4a2f-a100-d8deaa673950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:19.768345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4cd6d-6842-4a2f-a100-d8deaa673950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:19.770057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4cd6d-6842-4a2f-a100-d8deaa673950 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:19.781716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1051/2084] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:19.785920 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4cd6d-6842-4a2f-a100-d8deaa673950 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:19.801343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4cd6d-6842-4a2f-a100-d8deaa673950 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:19.811254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bfab6-b60e-43cd-bb95-f426a2ad8bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:19.811815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bfab6-b60e-43cd-bb95-f426a2ad8bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:19.812619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bfab6-b60e-43cd-bb95-f426a2ad8bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:19.812899 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bfab6-b60e-43cd-bb95-f426a2ad8bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:19.813616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bfab6-b60e-43cd-bb95-f426a2ad8bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:19.816510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bfab6-b60e-43cd-bb95-f426a2ad8bb2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:19.827656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1052/2085] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:50:19.831969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bfab6-b60e-43cd-bb95-f426a2ad8bb2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:19.842370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bfab6-b60e-43cd-bb95-f426a2ad8bb2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:19.847831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894309da-73d8-43f7-a586-e86c4b074f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:19.848485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894309da-73d8-43f7-a586-e86c4b074f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:19.848770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894309da-73d8-43f7-a586-e86c4b074f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:19.849054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894309da-73d8-43f7-a586-e86c4b074f91 admin admin] PATH_INFO: `/v3/projects/754b3ecf2c94466e9f107c16c5406f76/users/da60bef88a6249f99073077ae823cb96/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:19.866372 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894309da-73d8-43f7-a586-e86c4b074f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=754b3ecf2c94466e9f107c16c5406f76, user_id=da60bef88a6249f99073077ae823cb96, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:19.871167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894309da-73d8-43f7-a586-e86c4b074f91 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:19.878942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '051134557f784863b566fb376a71ef49', 'request_id': 'req-89b1a6fb-fd45-40d3-a444-6f489b814cd4'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.975586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '051134557f784863b566fb376a71ef49', 'request_id': 'req-89b1a6fb-fd45-40d3-a444-6f489b814cd4'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.984048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '051134557f784863b566fb376a71ef49', 'request_id': 'req-89b1a6fb-fd45-40d3-a444-6f489b814cd4'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:19.994920 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1053/2086] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:19 2022] PUT /identity/v3/projects/754b3ecf2c94466e9f107c16c5406f76/users/da60bef88a6249f99073077ae823cb96/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:20.004367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894309da-73d8-43f7-a586-e86c4b074f91 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:20.008297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-894309da-73d8-43f7-a586-e86c4b074f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:20.014204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] The token cache is being invalidate because project 051134557f784863b566fb376a71ef49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:20.014497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.015998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-89b1a6fb-fd45-40d3-a444-6f489b814cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.019255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1034/2087] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:19 2022] DELETE /identity/v3/projects/051134557f784863b566fb376a71ef49 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:20.024749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59dd0-8ee8-4dbc-a2a5-2711f1c4dd51 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:20.025036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59dd0-8ee8-4dbc-a2a5-2711f1c4dd51 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:20.038000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59dd0-8ee8-4dbc-a2a5-2711f1c4dd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:20.097204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-f7e59dd0-8ee8-4dbc-a2a5-2711f1c4dd51 None None] MFA Rules not processed for user `da60bef88a6249f99073077ae823cb96`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:20.098885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f7e59dd0-8ee8-4dbc-a2a5-2711f1c4dd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:20.166757 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894309da-73d8-43f7-a586-e86c4b074f91 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:20.172243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:20.172888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:20.174577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:20.175114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] PATH_INFO: `/v3/projects/9348eba198dd4d68ba069cc45ae1366c` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:20.175980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9348eba198dd4d68ba069cc45ae1366c)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:20.178096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:20.225691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1035/2088] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:20.226633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e59dd0-8ee8-4dbc-a2a5-2711f1c4dd51 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:20.227989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f7e59dd0-8ee8-4dbc-a2a5-2711f1c4dd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:20.252284 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e59dd0-8ee8-4dbc-a2a5-2711f1c4dd51 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:20.260481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:20.261125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:20.261848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:20.261848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] PATH_INFO: `/v3/users/8a4e89ad2e3a497b88c9ad0da1396b6d` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:20.263541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a4e89ad2e3a497b88c9ad0da1396b6d)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:20.267647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:20.360131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9348eba198dd4d68ba069cc45ae1366c', 'request_id': 'req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.368321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9348eba198dd4d68ba069cc45ae1366c', 'request_id': 'req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.383145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:20.455451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9348eba198dd4d68ba069cc45ae1366c', 'request_id': 'req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.467056 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] The token cache is being invalidate because project 9348eba198dd4d68ba069cc45ae1366c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:20.467342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.468003 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.469516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1054/2089] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:19 2022] DELETE /identity/v3/projects/9348eba198dd4d68ba069cc45ae1366c => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:20.470998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:20.472260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:20.521685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a4e89ad2e3a497b88c9ad0da1396b6d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.583995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9809e0-6c9d-478a-bed1-1778a8c6f72f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:20.590233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a1c2b-3b4f-4893-b4e3-c509f102fb90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:20.590860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a1c2b-3b4f-4893-b4e3-c509f102fb90 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:20.591147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a1c2b-3b4f-4893-b4e3-c509f102fb90 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:20.591421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a1c2b-3b4f-4893-b4e3-c509f102fb90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:20.593239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-919a1c2b-3b4f-4893-b4e3-c509f102fb90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:20.600084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a4e89ad2e3a497b88c9ad0da1396b6d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.601040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a4e89ad2e3a497b88c9ad0da1396b6d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.609541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a4e89ad2e3a497b88c9ad0da1396b6d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.622530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a2b9add6-4eda-4ad6-a8a5-209c56bdb7b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a4e89ad2e3a497b88c9ad0da1396b6d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:20.629000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1036/2090] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:20 2022] DELETE /identity/v3/users/8a4e89ad2e3a497b88c9ad0da1396b6d => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:20.630390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dd875-0805-47b8-8a21-d0d9224bc6f0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:20.630678 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dd875-0805-47b8-8a21-d0d9224bc6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:20.630953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dd875-0805-47b8-8a21-d0d9224bc6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:20.654116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4b8dd875-0805-47b8-8a21-d0d9224bc6f0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:20.656770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4b8dd875-0805-47b8-8a21-d0d9224bc6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:20.674888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a1c2b-3b4f-4893-b4e3-c509f102fb90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:20.677026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a1c2b-3b4f-4893-b4e3-c509f102fb90 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:20.688077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1055/2091] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:20.736347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1037/2092] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:20.744914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90d148-8017-45e0-b32d-93ad1454ec19 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:20.745201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90d148-8017-45e0-b32d-93ad1454ec19 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:20.745479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90d148-8017-45e0-b32d-93ad1454ec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:20.772351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3b90d148-8017-45e0-b32d-93ad1454ec19 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:20.773579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3b90d148-8017-45e0-b32d-93ad1454ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:20.784193 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1056/2093] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:20.788166 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8dd875-0805-47b8-8a21-d0d9224bc6f0 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:20.818167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8dd875-0805-47b8-8a21-d0d9224bc6f0 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:20.831511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ece2d3-7aea-4716-9233-53df65a9ef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:20.832410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ece2d3-7aea-4716-9233-53df65a9ef0a admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:20.832713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ece2d3-7aea-4716-9233-53df65a9ef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:20.832996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ece2d3-7aea-4716-9233-53df65a9ef0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:20.833654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ece2d3-7aea-4716-9233-53df65a9ef0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:50:20.834071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ece2d3-7aea-4716-9233-53df65a9ef0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:20.835884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ece2d3-7aea-4716-9233-53df65a9ef0a admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:20.850689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1038/2094] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:50:20.855033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90d148-8017-45e0-b32d-93ad1454ec19 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:20.865798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90d148-8017-45e0-b32d-93ad1454ec19 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:20.873135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6094d4-e83d-4e2b-b268-c7151cf9e2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:20.874202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6094d4-e83d-4e2b-b268-c7151cf9e2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:20.874689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6094d4-e83d-4e2b-b268-c7151cf9e2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:20.875155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6094d4-e83d-4e2b-b268-c7151cf9e2a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:20.876286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6094d4-e83d-4e2b-b268-c7151cf9e2a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:20.880011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6094d4-e83d-4e2b-b268-c7151cf9e2a2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:21.002631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ece2d3-7aea-4716-9233-53df65a9ef0a admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:21.004856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-16ece2d3-7aea-4716-9233-53df65a9ef0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:21.006721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1057/2095] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:20 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:21.010791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6094d4-e83d-4e2b-b268-c7151cf9e2a2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:21.020669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ece2d3-7aea-4716-9233-53df65a9ef0a admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:21.023894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:21.087280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:21.087910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:21.088189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:21.088462 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] PATH_INFO: `/v3/projects/9c9423d36b6540f99953a4598fa4feea` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:21.088888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6094d4-e83d-4e2b-b268-c7151cf9e2a2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:21.089592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9423d36b6540f99953a4598fa4feea)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:21.091574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:21.094049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af49ca-fa84-48e2-87f9-5077b6eb4c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:21.094655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af49ca-fa84-48e2-87f9-5077b6eb4c93 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:21.094956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af49ca-fa84-48e2-87f9-5077b6eb4c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:21.095221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af49ca-fa84-48e2-87f9-5077b6eb4c93 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:21.095934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af49ca-fa84-48e2-87f9-5077b6eb4c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:21.097874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af49ca-fa84-48e2-87f9-5077b6eb4c93 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:21.748816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1058/2096] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:21 2022] POST /identity/v3/users => generated 287 bytes in 739 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:21.753039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af49ca-fa84-48e2-87f9-5077b6eb4c93 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:21.762376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af49ca-fa84-48e2-87f9-5077b6eb4c93 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:21.766846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3551d3a-9823-43eb-87ec-d932d492e9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:21.767433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3551d3a-9823-43eb-87ec-d932d492e9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:21.767708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3551d3a-9823-43eb-87ec-d932d492e9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:21.767977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3551d3a-9823-43eb-87ec-d932d492e9c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:21.768597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3551d3a-9823-43eb-87ec-d932d492e9c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:21.770491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3551d3a-9823-43eb-87ec-d932d492e9c2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:21.782253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1059/2097] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:50:21.788433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3551d3a-9823-43eb-87ec-d932d492e9c2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:21.800832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3551d3a-9823-43eb-87ec-d932d492e9c2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:21.808570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9423d36b6540f99953a4598fa4feea', 'request_id': 'req-2f142bf4-efe2-4e5d-ad30-52fbdb114227'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:21.859880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d061d3-89a4-49dd-8c11-1d7b07411636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:21.860474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d061d3-89a4-49dd-8c11-1d7b07411636 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:21.860751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d061d3-89a4-49dd-8c11-1d7b07411636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:21.861029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d061d3-89a4-49dd-8c11-1d7b07411636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:21.861651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d061d3-89a4-49dd-8c11-1d7b07411636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:21.863255 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d061d3-89a4-49dd-8c11-1d7b07411636 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:21.878442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1060/2098] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:21.878442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d061d3-89a4-49dd-8c11-1d7b07411636 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:21.886727 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d061d3-89a4-49dd-8c11-1d7b07411636 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:21.891252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abc709-0e09-49fa-b269-b5553eaf9f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:21.891851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc709-0e09-49fa-b269-b5553eaf9f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:21.892127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc709-0e09-49fa-b269-b5553eaf9f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:21.892907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc709-0e09-49fa-b269-b5553eaf9f87 admin admin] PATH_INFO: `/v3/projects/ba65b0f6c70e4c5db317e5c673871a0a/users/fe3d3fe481e24a8d80dbb3e7d3e6a4d1/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:21.907611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abc709-0e09-49fa-b269-b5553eaf9f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba65b0f6c70e4c5db317e5c673871a0a, user_id=fe3d3fe481e24a8d80dbb3e7d3e6a4d1, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:21.911430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abc709-0e09-49fa-b269-b5553eaf9f87 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:21.923760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9423d36b6540f99953a4598fa4feea', 'request_id': 'req-2f142bf4-efe2-4e5d-ad30-52fbdb114227'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:21.931883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9423d36b6540f99953a4598fa4feea', 'request_id': 'req-2f142bf4-efe2-4e5d-ad30-52fbdb114227'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:21.944512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] The token cache is being invalidate because project 9c9423d36b6540f99953a4598fa4feea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:21.944799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:21.945455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2f142bf4-efe2-4e5d-ad30-52fbdb114227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:21.947003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1039/2099] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:21 2022] DELETE /identity/v3/projects/9c9423d36b6540f99953a4598fa4feea => generated 0 bytes in 946 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:21.951915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cfab3-080f-4c64-8697-e4aa1ccdd9cd None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:21.952096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cfab3-080f-4c64-8697-e4aa1ccdd9cd None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:21.952307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cfab3-080f-4c64-8697-e4aa1ccdd9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:21.973346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-cc6cfab3-080f-4c64-8697-e4aa1ccdd9cd None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:21.974569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-cc6cfab3-080f-4c64-8697-e4aa1ccdd9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:21.982530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1040/2100] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:50:21.991203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7fc2-968a-4846-9070-3158ddadc926 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:21.994132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7fc2-968a-4846-9070-3158ddadc926 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:21.994132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7fc2-968a-4846-9070-3158ddadc926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.012743 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a09e7fc2-968a-4846-9070-3158ddadc926 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:22.014119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a09e7fc2-968a-4846-9070-3158ddadc926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:22.034847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1061/2101] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:21 2022] PUT /identity/v3/projects/ba65b0f6c70e4c5db317e5c673871a0a/users/fe3d3fe481e24a8d80dbb3e7d3e6a4d1/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:22.045597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247a02c-8deb-4385-b69e-ce45c9fd6423 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.045919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247a02c-8deb-4385-b69e-ce45c9fd6423 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.046262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247a02c-8deb-4385-b69e-ce45c9fd6423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.082283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1041/2102] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:50:22.087286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e7fc2-968a-4846-9070-3158ddadc926 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:22.096874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8247a02c-8deb-4385-b69e-ce45c9fd6423 None None] MFA Rules not processed for user `fe3d3fe481e24a8d80dbb3e7d3e6a4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:22.098550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8247a02c-8deb-4385-b69e-ce45c9fd6423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:22.098639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e7fc2-968a-4846-9070-3158ddadc926 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:22.112998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf1207-6ac2-4d02-b297-ae03b0d8a5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:22.114111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf1207-6ac2-4d02-b297-ae03b0d8a5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.114400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf1207-6ac2-4d02-b297-ae03b0d8a5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.115072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf1207-6ac2-4d02-b297-ae03b0d8a5b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.115734 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf1207-6ac2-4d02-b297-ae03b0d8a5b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:50:22.116041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf1207-6ac2-4d02-b297-ae03b0d8a5b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:22.117693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf1207-6ac2-4d02-b297-ae03b0d8a5b2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:22.133282 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1042/2103] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:50:22.134218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf1207-6ac2-4d02-b297-ae03b0d8a5b2 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:22.145907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf1207-6ac2-4d02-b297-ae03b0d8a5b2 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:22.151209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbe299-ec8b-4383-b20f-c9f61d6994fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:22.151842 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbe299-ec8b-4383-b20f-c9f61d6994fb admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.152146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbe299-ec8b-4383-b20f-c9f61d6994fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.152449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbe299-ec8b-4383-b20f-c9f61d6994fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.153277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbe299-ec8b-4383-b20f-c9f61d6994fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:22.155455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbe299-ec8b-4383-b20f-c9f61d6994fb admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:22.197356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1062/2104] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:22.209952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247a02c-8deb-4385-b69e-ce45c9fd6423 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:22.213858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8247a02c-8deb-4385-b69e-ce45c9fd6423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:22.269331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247a02c-8deb-4385-b69e-ce45c9fd6423 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:22.273347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73766fb-96fd-4bd4-9d31-81a99711bd38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:22.273962 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73766fb-96fd-4bd4-9d31-81a99711bd38 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.274267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73766fb-96fd-4bd4-9d31-81a99711bd38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.274517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73766fb-96fd-4bd4-9d31-81a99711bd38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.277025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1043/2105] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:22 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:22.280736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbe299-ec8b-4383-b20f-c9f61d6994fb admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:22.290365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73766fb-96fd-4bd4-9d31-81a99711bd38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:22.290365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73766fb-96fd-4bd4-9d31-81a99711bd38 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:22.304217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1063/2106] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:50:22.338957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73766fb-96fd-4bd4-9d31-81a99711bd38 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:22.340939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b73766fb-96fd-4bd4-9d31-81a99711bd38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:22.355145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbe299-ec8b-4383-b20f-c9f61d6994fb admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:22.356308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73766fb-96fd-4bd4-9d31-81a99711bd38 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:22.360672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43982f5c-826b-4d30-8b73-febadad61b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:22.365067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43982f5c-826b-4d30-8b73-febadad61b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.365067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43982f5c-826b-4d30-8b73-febadad61b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.365067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43982f5c-826b-4d30-8b73-febadad61b5e admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.365335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:22.365442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43982f5c-826b-4d30-8b73-febadad61b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:22.365529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.365529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.365529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] PATH_INFO: `/v3/users/fe3d3fe481e24a8d80dbb3e7d3e6a4d1` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.365809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43982f5c-826b-4d30-8b73-febadad61b5e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:22.365891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3d3fe481e24a8d80dbb3e7d3e6a4d1)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:22.366602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:22.474496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1044/2107] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:22 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:22.478196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43982f5c-826b-4d30-8b73-febadad61b5e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:22.487846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43982f5c-826b-4d30-8b73-febadad61b5e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:22.492526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c31c16-290d-4909-bd5f-cfb26565299f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:22.492980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c31c16-290d-4909-bd5f-cfb26565299f admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.493259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c31c16-290d-4909-bd5f-cfb26565299f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.493530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c31c16-290d-4909-bd5f-cfb26565299f admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.494191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c31c16-290d-4909-bd5f-cfb26565299f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:22.495846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c31c16-290d-4909-bd5f-cfb26565299f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:22.506501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1045/2108] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:50:22.510482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c31c16-290d-4909-bd5f-cfb26565299f admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:22.520672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c31c16-290d-4909-bd5f-cfb26565299f admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:22.525253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567d91-25c7-42c8-8c29-221049525571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:22.525861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567d91-25c7-42c8-8c29-221049525571 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.526139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567d91-25c7-42c8-8c29-221049525571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.526416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567d91-25c7-42c8-8c29-221049525571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.527131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567d91-25c7-42c8-8c29-221049525571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:22.528816 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567d91-25c7-42c8-8c29-221049525571 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:22.540602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1046/2109] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:22.544649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567d91-25c7-42c8-8c29-221049525571 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:22.556111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567d91-25c7-42c8-8c29-221049525571 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:22.567033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4014f0a-b5c1-4211-8738-46d4f78c83a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:22.567731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014f0a-b5c1-4211-8738-46d4f78c83a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.568006 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014f0a-b5c1-4211-8738-46d4f78c83a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.568276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014f0a-b5c1-4211-8738-46d4f78c83a4 admin admin] PATH_INFO: `/v3/projects/5ae68e0d3ded46808ca91e5c35fa786e/users/b36a1afe219d44f8ac5a4b11d4f87b71/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.585170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4014f0a-b5c1-4211-8738-46d4f78c83a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae68e0d3ded46808ca91e5c35fa786e, user_id=b36a1afe219d44f8ac5a4b11d4f87b71, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:22.588257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4014f0a-b5c1-4211-8738-46d4f78c83a4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:22.647699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3d3fe481e24a8d80dbb3e7d3e6a4d1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:22.650009 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3d3fe481e24a8d80dbb3e7d3e6a4d1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:22.715996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1047/2110] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:22 2022] PUT /identity/v3/projects/5ae68e0d3ded46808ca91e5c35fa786e/users/b36a1afe219d44f8ac5a4b11d4f87b71/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:22.736666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ccb53-437e-4bf8-9727-7191669dcdb6 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.737315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ccb53-437e-4bf8-9727-7191669dcdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.737687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ccb53-437e-4bf8-9727-7191669dcdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.752722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3d3fe481e24a8d80dbb3e7d3e6a4d1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:22.754357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3d3fe481e24a8d80dbb3e7d3e6a4d1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:22.771297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-681e3ab5-5a37-49af-b03d-847cdcc6cbb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3d3fe481e24a8d80dbb3e7d3e6a4d1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:22.794264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1064/2111] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:22 2022] DELETE /identity/v3/users/fe3d3fe481e24a8d80dbb3e7d3e6a4d1 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:22.811096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b38a95-df23-49a5-b71f-c561cb79a2a6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.811368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b38a95-df23-49a5-b71f-c561cb79a2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.811641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b38a95-df23-49a5-b71f-c561cb79a2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.818931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b42ccb53-437e-4bf8-9727-7191669dcdb6 None None] MFA Rules not processed for user `b36a1afe219d44f8ac5a4b11d4f87b71`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:22.820197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b42ccb53-437e-4bf8-9727-7191669dcdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:22.843504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f2b38a95-df23-49a5-b71f-c561cb79a2a6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:22.845103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f2b38a95-df23-49a5-b71f-c561cb79a2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:22.915725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1048/2112] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:50:22.921764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ccb53-437e-4bf8-9727-7191669dcdb6 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:22.923328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b42ccb53-437e-4bf8-9727-7191669dcdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:22.930361 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1065/2113] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:22.940446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ccb53-437e-4bf8-9727-7191669dcdb6 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:22.946851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275e152-1135-42c8-9ea5-a2ed1881f16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:22.947466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275e152-1135-42c8-9ea5-a2ed1881f16d admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:22.947760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275e152-1135-42c8-9ea5-a2ed1881f16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:22.948038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275e152-1135-42c8-9ea5-a2ed1881f16d admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:22.948807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275e152-1135-42c8-9ea5-a2ed1881f16d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:22.950125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b38a95-df23-49a5-b71f-c561cb79a2a6 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:22.954812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275e152-1135-42c8-9ea5-a2ed1881f16d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:22.955006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f2b38a95-df23-49a5-b71f-c561cb79a2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:23.018006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b38a95-df23-49a5-b71f-c561cb79a2a6 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:23.019731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64737b-aa77-43b4-a316-f127a201e3cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:23.020343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64737b-aa77-43b4-a316-f127a201e3cd service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.020630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64737b-aa77-43b4-a316-f127a201e3cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.020907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64737b-aa77-43b4-a316-f127a201e3cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.030171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64737b-aa77-43b4-a316-f127a201e3cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:23.031599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64737b-aa77-43b4-a316-f127a201e3cd service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:23.043025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1066/2114] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:50:23.089312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1049/2115] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:22 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:23.096621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64737b-aa77-43b4-a316-f127a201e3cd service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:23.106617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64737b-aa77-43b4-a316-f127a201e3cd service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:23.165329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf0552-c692-4f48-8267-f4907db64b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:23.165971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf0552-c692-4f48-8267-f4907db64b07 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.166250 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf0552-c692-4f48-8267-f4907db64b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.166523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf0552-c692-4f48-8267-f4907db64b07 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.167185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf0552-c692-4f48-8267-f4907db64b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:23.169114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf0552-c692-4f48-8267-f4907db64b07 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:23.277549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1067/2116] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:23 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:23.278840 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275e152-1135-42c8-9ea5-a2ed1881f16d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:23.291090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275e152-1135-42c8-9ea5-a2ed1881f16d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:23.296485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f3efe-041b-42c0-a927-a91d677e90f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:23.297275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f3efe-041b-42c0-a927-a91d677e90f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.297666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f3efe-041b-42c0-a927-a91d677e90f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.298126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f3efe-041b-42c0-a927-a91d677e90f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.300009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f3efe-041b-42c0-a927-a91d677e90f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:23.302031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f3efe-041b-42c0-a927-a91d677e90f4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:23.314401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1050/2117] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:23.317197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf0552-c692-4f48-8267-f4907db64b07 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:23.328485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf0552-c692-4f48-8267-f4907db64b07 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:23.334331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f421bca-f4bc-470d-9fa3-a2a5eca9d55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:23.334949 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f421bca-f4bc-470d-9fa3-a2a5eca9d55b admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.335228 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f421bca-f4bc-470d-9fa3-a2a5eca9d55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.342157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f421bca-f4bc-470d-9fa3-a2a5eca9d55b admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.342157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f421bca-f4bc-470d-9fa3-a2a5eca9d55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:23.342157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f421bca-f4bc-470d-9fa3-a2a5eca9d55b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:23.349951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1068/2118] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:23.354088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f3efe-041b-42c0-a927-a91d677e90f4 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:23.366291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f3efe-041b-42c0-a927-a91d677e90f4 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:23.371311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bbdddc-76c0-46eb-a670-69e3af7dc8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:23.372043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbdddc-76c0-46eb-a670-69e3af7dc8b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.372368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbdddc-76c0-46eb-a670-69e3af7dc8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.372601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbdddc-76c0-46eb-a670-69e3af7dc8b0 admin admin] PATH_INFO: `/v3/projects/c555855139714fdfaed60aaa0f11a563/users/0760dc6091764d2b99ed5a3cc1018cd6/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.389852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bbdddc-76c0-46eb-a670-69e3af7dc8b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c555855139714fdfaed60aaa0f11a563, user_id=0760dc6091764d2b99ed5a3cc1018cd6, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:23.393344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bbdddc-76c0-46eb-a670-69e3af7dc8b0 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:23.516055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f421bca-f4bc-470d-9fa3-a2a5eca9d55b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:23.518426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8f421bca-f4bc-470d-9fa3-a2a5eca9d55b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:23.524160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1051/2119] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:23 2022] PUT /identity/v3/projects/c555855139714fdfaed60aaa0f11a563/users/0760dc6091764d2b99ed5a3cc1018cd6/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:23.529667 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734994a6-658c-4c56-95e7-e4d3939316e1 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.530214 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734994a6-658c-4c56-95e7-e4d3939316e1 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.530614 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734994a6-658c-4c56-95e7-e4d3939316e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.539082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f421bca-f4bc-470d-9fa3-a2a5eca9d55b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:23.596137 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-734994a6-658c-4c56-95e7-e4d3939316e1 None None] MFA Rules not processed for user `0760dc6091764d2b99ed5a3cc1018cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:23.597775 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-734994a6-658c-4c56-95e7-e4d3939316e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:23.616219 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:23.617309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.617703 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.618068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] PATH_INFO: `/v3/projects/ba65b0f6c70e4c5db317e5c673871a0a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.619303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba65b0f6c70e4c5db317e5c673871a0a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:23.621421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:23.678041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1052/2120] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:50:23.682378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734994a6-658c-4c56-95e7-e4d3939316e1 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:23.692891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734994a6-658c-4c56-95e7-e4d3939316e1 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:23.697485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f166a-2bb8-4c32-ac75-698ef7bda682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:23.702305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f166a-2bb8-4c32-ac75-698ef7bda682 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.702305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f166a-2bb8-4c32-ac75-698ef7bda682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.702305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f166a-2bb8-4c32-ac75-698ef7bda682 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.702305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f166a-2bb8-4c32-ac75-698ef7bda682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:23.702305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f166a-2bb8-4c32-ac75-698ef7bda682 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:23.766352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba65b0f6c70e4c5db317e5c673871a0a', 'request_id': 'req-159b62ad-9e5c-4db6-a484-b83b067ff81a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:23.773202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba65b0f6c70e4c5db317e5c673871a0a', 'request_id': 'req-159b62ad-9e5c-4db6-a484-b83b067ff81a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:23.793979 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1053/2121] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:23 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:23.795380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f166a-2bb8-4c32-ac75-698ef7bda682 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:23.866252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba65b0f6c70e4c5db317e5c673871a0a', 'request_id': 'req-159b62ad-9e5c-4db6-a484-b83b067ff81a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:23.867308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f166a-2bb8-4c32-ac75-698ef7bda682 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:23.872202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2df44-c04c-47ad-9153-28c71d020950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:23.872853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2df44-c04c-47ad-9153-28c71d020950 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.873121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2df44-c04c-47ad-9153-28c71d020950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.873390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2df44-c04c-47ad-9153-28c71d020950 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.874042 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2df44-c04c-47ad-9153-28c71d020950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:23.875941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2df44-c04c-47ad-9153-28c71d020950 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:23.879177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] The token cache is being invalidate because project ba65b0f6c70e4c5db317e5c673871a0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:23.879468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:23.885543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-159b62ad-9e5c-4db6-a484-b83b067ff81a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:23.887122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1069/2122] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:23 2022] DELETE /identity/v3/projects/ba65b0f6c70e4c5db317e5c673871a0a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:23.894090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dda70-d8c7-4d7a-9a47-455b7ff5eb18 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.894090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dda70-d8c7-4d7a-9a47-455b7ff5eb18 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.894498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dda70-d8c7-4d7a-9a47-455b7ff5eb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.895990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:23.916344 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9c0dda70-d8c7-4d7a-9a47-455b7ff5eb18 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:23.917611 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9c0dda70-d8c7-4d7a-9a47-455b7ff5eb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:23.926435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1070/2123] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:50:23.935142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab8b39-663c-4508-b9c5-9a90faea1527 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.935447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab8b39-663c-4508-b9c5-9a90faea1527 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.935725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab8b39-663c-4508-b9c5-9a90faea1527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.958378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-dbab8b39-663c-4508-b9c5-9a90faea1527 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:23.959515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dbab8b39-663c-4508-b9c5-9a90faea1527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:23.967967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1071/2124] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:50:23.972006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab8b39-663c-4508-b9c5-9a90faea1527 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:23.981087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab8b39-663c-4508-b9c5-9a90faea1527 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:23.986046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f747e10-0b3f-485d-b997-420542bb1689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:23.990276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f747e10-0b3f-485d-b997-420542bb1689 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:23.990276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f747e10-0b3f-485d-b997-420542bb1689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:23.990276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f747e10-0b3f-485d-b997-420542bb1689 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:23.990276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f747e10-0b3f-485d-b997-420542bb1689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:50:23.990276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f747e10-0b3f-485d-b997-420542bb1689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:23.990714 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f747e10-0b3f-485d-b997-420542bb1689 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:24.003197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1072/2125] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:50:24.007513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f747e10-0b3f-485d-b997-420542bb1689 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:24.018148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f747e10-0b3f-485d-b997-420542bb1689 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:24.021968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1054/2126] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:23 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:24.023227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2df44-c04c-47ad-9153-28c71d020950 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:24.024801 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b0dc0-35c7-409e-932b-5da1cabc433f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:24.025177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4ab2df44-c04c-47ad-9153-28c71d020950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:24.025468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b0dc0-35c7-409e-932b-5da1cabc433f admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.025831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b0dc0-35c7-409e-932b-5da1cabc433f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.026112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b0dc0-35c7-409e-932b-5da1cabc433f admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.030639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b0dc0-35c7-409e-932b-5da1cabc433f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:24.030639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b0dc0-35c7-409e-932b-5da1cabc433f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:24.041296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2df44-c04c-47ad-9153-28c71d020950 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:24.049980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb001dd9-bed6-49cd-9387-c4ac01b158a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:24.053747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb001dd9-bed6-49cd-9387-c4ac01b158a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.053747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb001dd9-bed6-49cd-9387-c4ac01b158a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.053747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb001dd9-bed6-49cd-9387-c4ac01b158a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.053747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb001dd9-bed6-49cd-9387-c4ac01b158a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:24.053747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb001dd9-bed6-49cd-9387-c4ac01b158a3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:24.068888 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1055/2127] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:50:24.073458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb001dd9-bed6-49cd-9387-c4ac01b158a3 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:24.084214 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb001dd9-bed6-49cd-9387-c4ac01b158a3 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:24.089528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b51752-ef6a-4455-a572-e50a992690b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:24.090253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b51752-ef6a-4455-a572-e50a992690b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.090533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b51752-ef6a-4455-a572-e50a992690b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.090813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b51752-ef6a-4455-a572-e50a992690b1 admin admin] PATH_INFO: `/v3/projects/981b5a46bff2436683d2fdf97754e062/users/be6cbb0ca30a4704af55da61381b7051/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.108670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b51752-ef6a-4455-a572-e50a992690b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=981b5a46bff2436683d2fdf97754e062, user_id=be6cbb0ca30a4704af55da61381b7051, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:24.112165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b51752-ef6a-4455-a572-e50a992690b1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:24.143758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1073/2128] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:24 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:24.147828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b0dc0-35c7-409e-932b-5da1cabc433f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:24.211379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b0dc0-35c7-409e-932b-5da1cabc433f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:24.216535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d9500-8259-451f-aad6-3d33ae5a468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:24.217139 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d9500-8259-451f-aad6-3d33ae5a468f admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.217497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d9500-8259-451f-aad6-3d33ae5a468f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.217799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d9500-8259-451f-aad6-3d33ae5a468f admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.218481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d9500-8259-451f-aad6-3d33ae5a468f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:24.220430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d9500-8259-451f-aad6-3d33ae5a468f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:24.369543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1056/2129] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:24 2022] PUT /identity/v3/projects/981b5a46bff2436683d2fdf97754e062/users/be6cbb0ca30a4704af55da61381b7051/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:24.373323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b51752-ef6a-4455-a572-e50a992690b1 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:24.406238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1074/2130] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:24 2022] POST /identity/v3/users => generated 282 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:50:24.413379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d9500-8259-451f-aad6-3d33ae5a468f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:24.425146 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d9500-8259-451f-aad6-3d33ae5a468f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:24.438383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b51752-ef6a-4455-a572-e50a992690b1 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:24.444181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9db0e-4532-4b06-aa3e-47d58064d6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:24.444792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9db0e-4532-4b06-aa3e-47d58064d6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.445074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9db0e-4532-4b06-aa3e-47d58064d6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.445686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9db0e-4532-4b06-aa3e-47d58064d6c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.446021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9db0e-4532-4b06-aa3e-47d58064d6c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:24.447634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9db0e-4532-4b06-aa3e-47d58064d6c9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:24.458358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1057/2131] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:24.462826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9db0e-4532-4b06-aa3e-47d58064d6c9 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:24.474081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9db0e-4532-4b06-aa3e-47d58064d6c9 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:24.478747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c8c45-2ddc-4429-a633-c0a553ae6a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:24.479391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c8c45-2ddc-4429-a633-c0a553ae6a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.479668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c8c45-2ddc-4429-a633-c0a553ae6a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.479935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c8c45-2ddc-4429-a633-c0a553ae6a86 admin admin] PATH_INFO: `/v3/projects/981b5a46bff2436683d2fdf97754e062/users/be6cbb0ca30a4704af55da61381b7051/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.482055 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c8c45-2ddc-4429-a633-c0a553ae6a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=981b5a46bff2436683d2fdf97754e062, user_id=be6cbb0ca30a4704af55da61381b7051, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:24.484847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c8c45-2ddc-4429-a633-c0a553ae6a86 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:24.486823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0b31c-ff33-4a06-8963-6f4699d9ad1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:24.487535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0b31c-ff33-4a06-8963-6f4699d9ad1b admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.487849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0b31c-ff33-4a06-8963-6f4699d9ad1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.488093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0b31c-ff33-4a06-8963-6f4699d9ad1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.488739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0b31c-ff33-4a06-8963-6f4699d9ad1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:24.490388 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0b31c-ff33-4a06-8963-6f4699d9ad1b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:24.497449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1058/2132] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:24 2022] PUT /identity/v3/projects/981b5a46bff2436683d2fdf97754e062/users/be6cbb0ca30a4704af55da61381b7051/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:24.502059 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1075/2133] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:50:24.514044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c8c45-2ddc-4429-a633-c0a553ae6a86 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:24.514157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7d768-28ac-42f7-abd1-3c17e96ae049 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.514157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7d768-28ac-42f7-abd1-3c17e96ae049 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.514157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7d768-28ac-42f7-abd1-3c17e96ae049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.541505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f4a7d768-28ac-42f7-abd1-3c17e96ae049 None None] MFA Rules not processed for user `be6cbb0ca30a4704af55da61381b7051`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:24.542790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f4a7d768-28ac-42f7-abd1-3c17e96ae049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:24.589897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c8c45-2ddc-4429-a633-c0a553ae6a86 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:24.604763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b91d6-75dd-4cbd-ab40-d9a878696c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:24.604763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b91d6-75dd-4cbd-ab40-d9a878696c31 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.604763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b91d6-75dd-4cbd-ab40-d9a878696c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.604763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b91d6-75dd-4cbd-ab40-d9a878696c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.606127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b91d6-75dd-4cbd-ab40-d9a878696c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:24.606127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b91d6-75dd-4cbd-ab40-d9a878696c31 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:24.611828 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1059/2134] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:24.615900 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b91d6-75dd-4cbd-ab40-d9a878696c31 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:24.627546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b91d6-75dd-4cbd-ab40-d9a878696c31 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:24.629363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:24.631534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1076/2135] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:50:24.638768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3889b3-3029-403f-b974-af29ca120ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:24.638768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3889b3-3029-403f-b974-af29ca120ffd admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.638768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3889b3-3029-403f-b974-af29ca120ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.638768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3889b3-3029-403f-b974-af29ca120ffd admin admin] PATH_INFO: `/v3/projects/d27f28ed6ca843178174ceeaa12be4bc/users/693d8e1548e348c0aefe8ef7feb1f70c/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.654366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3889b3-3029-403f-b974-af29ca120ffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27f28ed6ca843178174ceeaa12be4bc, user_id=693d8e1548e348c0aefe8ef7feb1f70c, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:24.657086 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3889b3-3029-403f-b974-af29ca120ffd admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:24.797933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1060/2136] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:24 2022] PUT /identity/v3/projects/d27f28ed6ca843178174ceeaa12be4bc/users/693d8e1548e348c0aefe8ef7feb1f70c/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:24.808538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927beaa7-9331-4964-aa26-ff95e64b38cc None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:24.808933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927beaa7-9331-4964-aa26-ff95e64b38cc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:24.809083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927beaa7-9331-4964-aa26-ff95e64b38cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:24.837496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-927beaa7-9331-4964-aa26-ff95e64b38cc None None] MFA Rules not processed for user `693d8e1548e348c0aefe8ef7feb1f70c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:24.838768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-927beaa7-9331-4964-aa26-ff95e64b38cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:24.904870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1077/2137] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:25.175529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3889b3-3029-403f-b974-af29ca120ffd admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:25.177333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-af3889b3-3029-403f-b974-af29ca120ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:25.233178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3889b3-3029-403f-b974-af29ca120ffd admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:25.236769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddcf58-cdb7-4850-a71c-f0809af96af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:25.237356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddcf58-cdb7-4850-a71c-f0809af96af6 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:25.241189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddcf58-cdb7-4850-a71c-f0809af96af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:25.241189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddcf58-cdb7-4850-a71c-f0809af96af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:25.241189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-64ddcf58-cdb7-4850-a71c-f0809af96af6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:25.302560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ddcf58-cdb7-4850-a71c-f0809af96af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:25.303858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ddcf58-cdb7-4850-a71c-f0809af96af6 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:25.315443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927beaa7-9331-4964-aa26-ff95e64b38cc None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:25.318181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1061/2138] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:25.320672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-927beaa7-9331-4964-aa26-ff95e64b38cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:25.335082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddcf58-cdb7-4850-a71c-f0809af96af6 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:25.336581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-64ddcf58-cdb7-4850-a71c-f0809af96af6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:25.337435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927beaa7-9331-4964-aa26-ff95e64b38cc None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:25.342529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26650984-10f2-4e81-95fe-99181c2fc565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:25.343607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650984-10f2-4e81-95fe-99181c2fc565 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:25.343960 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650984-10f2-4e81-95fe-99181c2fc565 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:25.344391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650984-10f2-4e81-95fe-99181c2fc565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:25.355834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26650984-10f2-4e81-95fe-99181c2fc565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:25.357589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26650984-10f2-4e81-95fe-99181c2fc565 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:25.374072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1078/2139] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:50:25.425420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddcf58-cdb7-4850-a71c-f0809af96af6 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:25.434260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:25.434966 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:25.435247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:25.435520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] PATH_INFO: `/v3/users/da60bef88a6249f99073077ae823cb96` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:25.437349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=da60bef88a6249f99073077ae823cb96)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:25.440440 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:25.639351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da60bef88a6249f99073077ae823cb96'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:25.729650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da60bef88a6249f99073077ae823cb96'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:25.730455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da60bef88a6249f99073077ae823cb96'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:25.738048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da60bef88a6249f99073077ae823cb96'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:25.750447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da60bef88a6249f99073077ae823cb96'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:25.756969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1062/2140] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:25 2022] DELETE /identity/v3/users/da60bef88a6249f99073077ae823cb96 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:25.761237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a9a6d-5f10-4b62-a211-dc48de3bbff4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:25.761525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a9a6d-5f10-4b62-a211-dc48de3bbff4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:25.761833 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a9a6d-5f10-4b62-a211-dc48de3bbff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:25.782128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4a1a9a6d-5f10-4b62-a211-dc48de3bbff4 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:25.784048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4a1a9a6d-5f10-4b62-a211-dc48de3bbff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:25.844600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1079/2141] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:25.855088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:25.856499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:25.914277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ae3b8-d143-492a-8ced-9748b4a659aa admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:25.918061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf5adf-9376-4a50-b0e3-0a29c80c16dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:25.918573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5adf-9376-4a50-b0e3-0a29c80c16dd service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:25.918851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5adf-9376-4a50-b0e3-0a29c80c16dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:25.919142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5adf-9376-4a50-b0e3-0a29c80c16dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:25.929019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf5adf-9376-4a50-b0e3-0a29c80c16dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:25.930468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf5adf-9376-4a50-b0e3-0a29c80c16dd service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:25.940323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1063/2142] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:26.392416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:26.509933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a9a6d-5f10-4b62-a211-dc48de3bbff4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:26.510879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4a1a9a6d-5f10-4b62-a211-dc48de3bbff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:26.524494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a9a6d-5f10-4b62-a211-dc48de3bbff4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:26.529205 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:26.529874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:26.530162 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:26.530436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] PATH_INFO: `/v3/projects/754b3ecf2c94466e9f107c16c5406f76` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:26.531570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=754b3ecf2c94466e9f107c16c5406f76)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:26.538434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:26.661216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '754b3ecf2c94466e9f107c16c5406f76', 'request_id': 'req-d0f8836c-4de4-457f-8752-9aa2402d9c97'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:26.667891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '754b3ecf2c94466e9f107c16c5406f76', 'request_id': 'req-d0f8836c-4de4-457f-8752-9aa2402d9c97'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:26.723193 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:26.751305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '754b3ecf2c94466e9f107c16c5406f76', 'request_id': 'req-d0f8836c-4de4-457f-8752-9aa2402d9c97'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:26.762372 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] The token cache is being invalidate because project 754b3ecf2c94466e9f107c16c5406f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:26.762663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:26.763303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d0f8836c-4de4-457f-8752-9aa2402d9c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:26.765078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1080/2143] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:26 2022] DELETE /identity/v3/projects/754b3ecf2c94466e9f107c16c5406f76 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:26.770216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c76b47-0216-4698-b230-4360f658cd75 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:26.770500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c76b47-0216-4698-b230-4360f658cd75 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:26.770772 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c76b47-0216-4698-b230-4360f658cd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:26.792603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-10c76b47-0216-4698-b230-4360f658cd75 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:26.793882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-10c76b47-0216-4698-b230-4360f658cd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:26.854562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1064/2144] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:26.864046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d365a5-1781-47d5-a516-fe3ce5b22514 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:26.864539 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d365a5-1781-47d5-a516-fe3ce5b22514 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:26.868013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d365a5-1781-47d5-a516-fe3ce5b22514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:26.884821 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b1d365a5-1781-47d5-a516-fe3ce5b22514 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:26.886107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b1d365a5-1781-47d5-a516-fe3ce5b22514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:26.894180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1081/2145] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:50:26.898519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c76b47-0216-4698-b230-4360f658cd75 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:26.908650 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c76b47-0216-4698-b230-4360f658cd75 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:26.913263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16203d-e294-49d8-a3a3-26883cd28dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:26.913940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16203d-e294-49d8-a3a3-26883cd28dea admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:26.914177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16203d-e294-49d8-a3a3-26883cd28dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:26.914449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16203d-e294-49d8-a3a3-26883cd28dea admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:26.915121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16203d-e294-49d8-a3a3-26883cd28dea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:50:26.915423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16203d-e294-49d8-a3a3-26883cd28dea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:26.917020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16203d-e294-49d8-a3a3-26883cd28dea admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:26.929648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1065/2146] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:50:26.933869 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d365a5-1781-47d5-a516-fe3ce5b22514 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:26.943317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d365a5-1781-47d5-a516-fe3ce5b22514 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:26.948239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf4e85-b16a-41de-957a-cfdb145db1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:26.948843 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf4e85-b16a-41de-957a-cfdb145db1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:26.949127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf4e85-b16a-41de-957a-cfdb145db1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:26.949406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf4e85-b16a-41de-957a-cfdb145db1b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:26.950097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf4e85-b16a-41de-957a-cfdb145db1b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:26.952021 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf4e85-b16a-41de-957a-cfdb145db1b7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:27.039468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1082/2147] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:26 2022] POST /identity/v3/projects => generated 355 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:27.043712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16203d-e294-49d8-a3a3-26883cd28dea admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:27.054674 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16203d-e294-49d8-a3a3-26883cd28dea admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:27.112178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc0d4a-307e-455a-a96f-0ce18b5bfbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:27.112784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc0d4a-307e-455a-a96f-0ce18b5bfbaf admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:27.113063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc0d4a-307e-455a-a96f-0ce18b5bfbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:27.113335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc0d4a-307e-455a-a96f-0ce18b5bfbaf admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:27.113991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc0d4a-307e-455a-a96f-0ce18b5bfbaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:27.115927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc0d4a-307e-455a-a96f-0ce18b5bfbaf admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:27.263393 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1066/2148] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:27 2022] POST /identity/v3/users => generated 278 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:27.268162 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf4e85-b16a-41de-957a-cfdb145db1b7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:27.278477 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf4e85-b16a-41de-957a-cfdb145db1b7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:27.283569 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e5d40-d196-49a6-91e0-58229f3925c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:27.284304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e5d40-d196-49a6-91e0-58229f3925c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:27.284578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e5d40-d196-49a6-91e0-58229f3925c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:27.284848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e5d40-d196-49a6-91e0-58229f3925c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:27.285482 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e5d40-d196-49a6-91e0-58229f3925c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:27.287098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e5d40-d196-49a6-91e0-58229f3925c4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:27.296984 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1083/2149] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:27.302182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc0d4a-307e-455a-a96f-0ce18b5bfbaf admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:27.312245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc0d4a-307e-455a-a96f-0ce18b5bfbaf admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:27.317182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e300dcb-7836-4e0d-8847-580a9805bc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:27.317806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e300dcb-7836-4e0d-8847-580a9805bc5e admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:27.318092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e300dcb-7836-4e0d-8847-580a9805bc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:27.318369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e300dcb-7836-4e0d-8847-580a9805bc5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:27.319011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e300dcb-7836-4e0d-8847-580a9805bc5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:27.320621 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e300dcb-7836-4e0d-8847-580a9805bc5e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:27.331581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1067/2150] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:27.336147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e5d40-d196-49a6-91e0-58229f3925c4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:27.347077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e5d40-d196-49a6-91e0-58229f3925c4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:27.353622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2af263-51e5-4f4d-95fe-8cd0a8d57abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:27.353622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2af263-51e5-4f4d-95fe-8cd0a8d57abe admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:27.353622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2af263-51e5-4f4d-95fe-8cd0a8d57abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:27.353622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2af263-51e5-4f4d-95fe-8cd0a8d57abe admin admin] PATH_INFO: `/v3/projects/0e0f154b85d043b39eddbc92f7642c30/users/2a01865a4cc34624ade14242ee521a08/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:27.373122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2af263-51e5-4f4d-95fe-8cd0a8d57abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e0f154b85d043b39eddbc92f7642c30, user_id=2a01865a4cc34624ade14242ee521a08, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:27.375865 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2af263-51e5-4f4d-95fe-8cd0a8d57abe admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:27.515225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1084/2151] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:27 2022] PUT /identity/v3/projects/0e0f154b85d043b39eddbc92f7642c30/users/2a01865a4cc34624ade14242ee521a08/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:27.530926 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4b787-e43a-451f-99d0-62cbe3e3bc33 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:27.531414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4b787-e43a-451f-99d0-62cbe3e3bc33 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:27.532347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4b787-e43a-451f-99d0-62cbe3e3bc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:27.585882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4ed4b787-e43a-451f-99d0-62cbe3e3bc33 None None] MFA Rules not processed for user `2a01865a4cc34624ade14242ee521a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:27.587569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4ed4b787-e43a-451f-99d0-62cbe3e3bc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:27.682941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1068/2152] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:27.687535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2af263-51e5-4f4d-95fe-8cd0a8d57abe admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:27.750171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4b787-e43a-451f-99d0-62cbe3e3bc33 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:27.750171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4ed4b787-e43a-451f-99d0-62cbe3e3bc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:27.769828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2af263-51e5-4f4d-95fe-8cd0a8d57abe admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:27.778561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c674ef4-1504-438a-b394-245a5906664e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:27.779267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c674ef4-1504-438a-b394-245a5906664e admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:27.779602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c674ef4-1504-438a-b394-245a5906664e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:27.779882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c674ef4-1504-438a-b394-245a5906664e admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:27.780524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c674ef4-1504-438a-b394-245a5906664e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:27.782668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c674ef4-1504-438a-b394-245a5906664e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:27.837434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4b787-e43a-451f-99d0-62cbe3e3bc33 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:27.841090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7785d-6e73-4132-bd6b-e87fd1935851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:27.841688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7785d-6e73-4132-bd6b-e87fd1935851 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:27.842030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7785d-6e73-4132-bd6b-e87fd1935851 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:27.842296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7785d-6e73-4132-bd6b-e87fd1935851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:27.844131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-17a7785d-6e73-4132-bd6b-e87fd1935851 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:27.908670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7785d-6e73-4132-bd6b-e87fd1935851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:27.910182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7785d-6e73-4132-bd6b-e87fd1935851 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:27.921042 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1069/2153] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:50:27.925395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1085/2154] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:27 2022] POST /identity/v3/projects => generated 357 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:27.933311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7785d-6e73-4132-bd6b-e87fd1935851 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:27.946468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7785d-6e73-4132-bd6b-e87fd1935851 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:28.019526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309cdcb-ad17-4c46-b9d4-5d7d886aee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:28.020266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309cdcb-ad17-4c46-b9d4-5d7d886aee5f admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:28.020741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309cdcb-ad17-4c46-b9d4-5d7d886aee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:28.021012 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309cdcb-ad17-4c46-b9d4-5d7d886aee5f admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:28.021820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309cdcb-ad17-4c46-b9d4-5d7d886aee5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:28.024141 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309cdcb-ad17-4c46-b9d4-5d7d886aee5f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:28.131131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1070/2155] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:27 2022] POST /identity/v3/users => generated 285 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:28.136324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c674ef4-1504-438a-b394-245a5906664e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:28.146730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c674ef4-1504-438a-b394-245a5906664e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:28.151473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00dabf-b770-4760-9741-28d9a0acb23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:28.152149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00dabf-b770-4760-9741-28d9a0acb23c admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:28.152520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00dabf-b770-4760-9741-28d9a0acb23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:28.152796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00dabf-b770-4760-9741-28d9a0acb23c admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:28.153434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00dabf-b770-4760-9741-28d9a0acb23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:28.155075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00dabf-b770-4760-9741-28d9a0acb23c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:28.170157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1086/2156] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:28.175236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309cdcb-ad17-4c46-b9d4-5d7d886aee5f admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:28.185582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309cdcb-ad17-4c46-b9d4-5d7d886aee5f admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:28.192636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5404fc0-e8c8-4d3e-b175-43895aeca824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:28.201931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5404fc0-e8c8-4d3e-b175-43895aeca824 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:28.201931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5404fc0-e8c8-4d3e-b175-43895aeca824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:28.201931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5404fc0-e8c8-4d3e-b175-43895aeca824 admin admin] PATH_INFO: `/v3/projects/01a0694f564d4450a0b5808cade74259/users/044c3ecc3b4b41c4a5dfa6dd18c327c0/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:28.208883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5404fc0-e8c8-4d3e-b175-43895aeca824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a0694f564d4450a0b5808cade74259, user_id=044c3ecc3b4b41c4a5dfa6dd18c327c0, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:28.211584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5404fc0-e8c8-4d3e-b175-43895aeca824 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:28.220692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:28.290099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1071/2157] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:28 2022] PUT /identity/v3/projects/01a0694f564d4450a0b5808cade74259/users/044c3ecc3b4b41c4a5dfa6dd18c327c0/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:28.294108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00dabf-b770-4760-9741-28d9a0acb23c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:28.304987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00dabf-b770-4760-9741-28d9a0acb23c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:28.365929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a3095-6871-4bfc-813b-5d7da6a82cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:28.366543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3095-6871-4bfc-813b-5d7da6a82cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:28.366822 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3095-6871-4bfc-813b-5d7da6a82cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:28.367097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3095-6871-4bfc-813b-5d7da6a82cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:28.367742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a3095-6871-4bfc-813b-5d7da6a82cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:28.369337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a3095-6871-4bfc-813b-5d7da6a82cfc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:28.384023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1087/2158] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:28.384926 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5404fc0-e8c8-4d3e-b175-43895aeca824 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:28.395464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5404fc0-e8c8-4d3e-b175-43895aeca824 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:28.400560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f67a6-19a6-4ac9-8726-c591164e790b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:28.401221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f67a6-19a6-4ac9-8726-c591164e790b admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:28.401517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f67a6-19a6-4ac9-8726-c591164e790b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:28.401843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f67a6-19a6-4ac9-8726-c591164e790b admin admin] PATH_INFO: `/v3/projects/01a0694f564d4450a0b5808cade74259/users/044c3ecc3b4b41c4a5dfa6dd18c327c0/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:28.403906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f67a6-19a6-4ac9-8726-c591164e790b admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a0694f564d4450a0b5808cade74259, user_id=044c3ecc3b4b41c4a5dfa6dd18c327c0, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:28.406720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f67a6-19a6-4ac9-8726-c591164e790b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:28.418933 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1072/2159] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:28 2022] PUT /identity/v3/projects/01a0694f564d4450a0b5808cade74259/users/044c3ecc3b4b41c4a5dfa6dd18c327c0/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:28.431743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605e266-4dbb-407f-9a5d-de85965d0767 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:28.432038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605e266-4dbb-407f-9a5d-de85965d0767 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:28.432327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605e266-4dbb-407f-9a5d-de85965d0767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:28.463016 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5605e266-4dbb-407f-9a5d-de85965d0767 None None] MFA Rules not processed for user `044c3ecc3b4b41c4a5dfa6dd18c327c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:28.464809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5605e266-4dbb-407f-9a5d-de85965d0767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:28.538496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1088/2160] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:28.605390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:28.623176 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f67a6-19a6-4ac9-8726-c591164e790b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:28.698040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f67a6-19a6-4ac9-8726-c591164e790b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:28.702837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579141e9-0d86-4de4-a9c7-2fde904d8a85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:28.703647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579141e9-0d86-4de4-a9c7-2fde904d8a85 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:28.704324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579141e9-0d86-4de4-a9c7-2fde904d8a85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:28.705288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579141e9-0d86-4de4-a9c7-2fde904d8a85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:28.719714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579141e9-0d86-4de4-a9c7-2fde904d8a85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:28.722827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579141e9-0d86-4de4-a9c7-2fde904d8a85 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:28.738058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1073/2161] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:28.875110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605e266-4dbb-407f-9a5d-de85965d0767 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:28.886371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605e266-4dbb-407f-9a5d-de85965d0767 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:28.890120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05feae-c399-4e09-a1a0-5846ec31c89f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:28.890741 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05feae-c399-4e09-a1a0-5846ec31c89f service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:28.891040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05feae-c399-4e09-a1a0-5846ec31c89f service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:28.891315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05feae-c399-4e09-a1a0-5846ec31c89f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:28.893973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4e05feae-c399-4e09-a1a0-5846ec31c89f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:28.968944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05feae-c399-4e09-a1a0-5846ec31c89f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:28.971170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05feae-c399-4e09-a1a0-5846ec31c89f service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:28.979965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1089/2162] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:29.758258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579141e9-0d86-4de4-a9c7-2fde904d8a85 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:29.759659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-579141e9-0d86-4de4-a9c7-2fde904d8a85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:29.770783 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579141e9-0d86-4de4-a9c7-2fde904d8a85 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:29.774899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737f02-be48-4b13-9953-8bf2421a58e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:29.775546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737f02-be48-4b13-9953-8bf2421a58e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:29.775826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737f02-be48-4b13-9953-8bf2421a58e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:29.776234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737f02-be48-4b13-9953-8bf2421a58e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:29.840043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2737f02-be48-4b13-9953-8bf2421a58e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:29.841525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2737f02-be48-4b13-9953-8bf2421a58e3 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:29.850992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1074/2163] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:30.137915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:30.828977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:32.225592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05feae-c399-4e09-a1a0-5846ec31c89f service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:32.234520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05feae-c399-4e09-a1a0-5846ec31c89f service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:32.238715 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463fe51-e27b-4503-93ce-d3a8cec3ac5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:32.239343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463fe51-e27b-4503-93ce-d3a8cec3ac5c service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:32.239585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463fe51-e27b-4503-93ce-d3a8cec3ac5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:32.239856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463fe51-e27b-4503-93ce-d3a8cec3ac5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:32.241604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4463fe51-e27b-4503-93ce-d3a8cec3ac5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:32.308749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463fe51-e27b-4503-93ce-d3a8cec3ac5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:32.310942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463fe51-e27b-4503-93ce-d3a8cec3ac5c service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:32.316936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:32.323492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1090/2164] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:32.439965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:33.103075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:33.492687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:35.193705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:35.512672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:36.450590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737f02-be48-4b13-9953-8bf2421a58e3 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:36.452105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a2737f02-be48-4b13-9953-8bf2421a58e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:36.514684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:36.555867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737f02-be48-4b13-9953-8bf2421a58e3 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:36.562290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:36.562930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:36.563236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:36.563697 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] PATH_INFO: `/v3/users/b85f61e5aa5140059d88dc49996e57ed` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:36.565120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b85f61e5aa5140059d88dc49996e57ed)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:36.567578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:36.794137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:36.840853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b85f61e5aa5140059d88dc49996e57ed'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:36.948774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b85f61e5aa5140059d88dc49996e57ed'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:36.949404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b85f61e5aa5140059d88dc49996e57ed'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:36.956993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b85f61e5aa5140059d88dc49996e57ed'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:36.969908 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b85f61e5aa5140059d88dc49996e57ed'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:36.976599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1075/2165] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:36 2022] DELETE /identity/v3/users/b85f61e5aa5140059d88dc49996e57ed => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:36.978166 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:36.981202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e51e9-87ee-43b3-b1e9-362eb236a371 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:36.981473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e51e9-87ee-43b3-b1e9-362eb236a371 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:36.981941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e51e9-87ee-43b3-b1e9-362eb236a371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:37.006568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-927e51e9-87ee-43b3-b1e9-362eb236a371 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:37.007777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-927e51e9-87ee-43b3-b1e9-362eb236a371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:37.074665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1091/2166] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:37.084760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:37.086222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:37.155312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c9d1f-5ade-49dc-90b2-3a28d48128b1 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:37.159033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace182e-08dc-4a62-b788-d911c948a2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:37.159635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace182e-08dc-4a62-b788-d911c948a2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:37.159912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace182e-08dc-4a62-b788-d911c948a2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:37.160182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace182e-08dc-4a62-b788-d911c948a2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:37.170167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace182e-08dc-4a62-b788-d911c948a2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:37.171601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace182e-08dc-4a62-b788-d911c948a2e5 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:37.182120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1076/2167] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:37.688600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e51e9-87ee-43b3-b1e9-362eb236a371 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:37.691592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-927e51e9-87ee-43b3-b1e9-362eb236a371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:37.705958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e51e9-87ee-43b3-b1e9-362eb236a371 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:37.711789 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:37.712723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:37.713039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:37.713319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] PATH_INFO: `/v3/projects/463a97b3425a4ada9253dce1047b29e8` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:37.714552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=463a97b3425a4ada9253dce1047b29e8)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:37.716502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:37.821521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:37.871621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '463a97b3425a4ada9253dce1047b29e8', 'request_id': 'req-73204477-b4ef-4f36-b7ae-a324a319b2ee'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:37.877866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '463a97b3425a4ada9253dce1047b29e8', 'request_id': 'req-73204477-b4ef-4f36-b7ae-a324a319b2ee'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:38.016651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '463a97b3425a4ada9253dce1047b29e8', 'request_id': 'req-73204477-b4ef-4f36-b7ae-a324a319b2ee'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:38.016985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:38.028225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] The token cache is being invalidate because project 463a97b3425a4ada9253dce1047b29e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:38.028513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:38.029138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:38.030769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1092/2168] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:37 2022] DELETE /identity/v3/projects/463a97b3425a4ada9253dce1047b29e8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:38.034584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace182e-08dc-4a62-b788-d911c948a2e5 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:38.044322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace182e-08dc-4a62-b788-d911c948a2e5 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:38.101511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:38.102338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:38.102620 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:38.102900 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] PATH_INFO: `/v3/users/24e73cdc80594548bd1002ca28be8061` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:38.104033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e73cdc80594548bd1002ca28be8061)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:38.105986 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:38.308306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e73cdc80594548bd1002ca28be8061'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:38.387957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e73cdc80594548bd1002ca28be8061'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:38.388580 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e73cdc80594548bd1002ca28be8061'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:38.395855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e73cdc80594548bd1002ca28be8061'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:38.409037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-64c8ea30-4d88-45fa-9d22-f8c6938af2c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e73cdc80594548bd1002ca28be8061'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:38.416199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1077/2169] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:38 2022] DELETE /identity/v3/users/24e73cdc80594548bd1002ca28be8061 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:38.759796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:38.959338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:38.960784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:38.974576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73204477-b4ef-4f36-b7ae-a324a319b2ee admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:39.036004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:39.036543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:39.036874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:39.037148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] PATH_INFO: `/v3/projects/24b8780568c14aeb8acdfe076c6fbefb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:39.038425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24b8780568c14aeb8acdfe076c6fbefb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:39.040312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:39.198584 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:39.216880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24b8780568c14aeb8acdfe076c6fbefb', 'request_id': 'req-310fef0c-2045-4e7c-8441-d674826540a4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:39.222981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24b8780568c14aeb8acdfe076c6fbefb', 'request_id': 'req-310fef0c-2045-4e7c-8441-d674826540a4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:39.311331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24b8780568c14aeb8acdfe076c6fbefb', 'request_id': 'req-310fef0c-2045-4e7c-8441-d674826540a4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:39.323128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] The token cache is being invalidate because project 24b8780568c14aeb8acdfe076c6fbefb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:39.324490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:39.324490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-310fef0c-2045-4e7c-8441-d674826540a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:39.325526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1093/2170] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:38 2022] DELETE /identity/v3/projects/24b8780568c14aeb8acdfe076c6fbefb => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:39.331822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa38c65-d47b-428f-a50a-11f83c1e0a24 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:39.332294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa38c65-d47b-428f-a50a-11f83c1e0a24 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:39.332439 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa38c65-d47b-428f-a50a-11f83c1e0a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:39.352917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-baa38c65-d47b-428f-a50a-11f83c1e0a24 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:39.354140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-baa38c65-d47b-428f-a50a-11f83c1e0a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:39.413496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1078/2171] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:39.421789 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f81a5-3148-405e-81ab-106613f86450 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:39.422081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f81a5-3148-405e-81ab-106613f86450 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:39.422361 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f81a5-3148-405e-81ab-106613f86450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:39.442533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ac4f81a5-3148-405e-81ab-106613f86450 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:39.443914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ac4f81a5-3148-405e-81ab-106613f86450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:39.451826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1094/2172] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:39.456102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa38c65-d47b-428f-a50a-11f83c1e0a24 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:39.466576 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa38c65-d47b-428f-a50a-11f83c1e0a24 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:39.471176 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b38ca9-24e8-41df-92e9-84c4b6259d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:39.471763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b38ca9-24e8-41df-92e9-84c4b6259d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:39.472038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b38ca9-24e8-41df-92e9-84c4b6259d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:39.472310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b38ca9-24e8-41df-92e9-84c4b6259d42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:39.472945 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b38ca9-24e8-41df-92e9-84c4b6259d42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:50:39.473342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b38ca9-24e8-41df-92e9-84c4b6259d42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:39.475702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b38ca9-24e8-41df-92e9-84c4b6259d42 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:39.487058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1079/2173] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:50:39.492260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f81a5-3148-405e-81ab-106613f86450 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:39.501420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f81a5-3148-405e-81ab-106613f86450 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:39.506642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3bb80-c736-405c-a190-6e419a0ee257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:39.507238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3bb80-c736-405c-a190-6e419a0ee257 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:39.507514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3bb80-c736-405c-a190-6e419a0ee257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:39.507790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3bb80-c736-405c-a190-6e419a0ee257 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:39.508421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3bb80-c736-405c-a190-6e419a0ee257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:39.510370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3bb80-c736-405c-a190-6e419a0ee257 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:39.608872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1095/2174] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:39 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:39.612394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b38ca9-24e8-41df-92e9-84c4b6259d42 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:39.623102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b38ca9-24e8-41df-92e9-84c4b6259d42 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:39.671561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:39.684744 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3ed84-aa5d-4c7f-b857-ae8a5f735237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:39.685343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3ed84-aa5d-4c7f-b857-ae8a5f735237 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:39.685619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3ed84-aa5d-4c7f-b857-ae8a5f735237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:39.685922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3ed84-aa5d-4c7f-b857-ae8a5f735237 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:39.686628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3ed84-aa5d-4c7f-b857-ae8a5f735237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:39.688544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3ed84-aa5d-4c7f-b857-ae8a5f735237 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:39.768786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:39.811452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1080/2175] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:39 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:39.818748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3bb80-c736-405c-a190-6e419a0ee257 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:39.826378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3bb80-c736-405c-a190-6e419a0ee257 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:39.830854 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adf95c-0d33-441b-84f9-e8792afa03f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:39.831438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adf95c-0d33-441b-84f9-e8792afa03f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:39.831716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adf95c-0d33-441b-84f9-e8792afa03f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:39.832020 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adf95c-0d33-441b-84f9-e8792afa03f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:39.832683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adf95c-0d33-441b-84f9-e8792afa03f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:39.834558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adf95c-0d33-441b-84f9-e8792afa03f9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:39.844695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1096/2176] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:39.853056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3ed84-aa5d-4c7f-b857-ae8a5f735237 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:39.863084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3ed84-aa5d-4c7f-b857-ae8a5f735237 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:39.868395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61c242-3b5a-43d9-880f-4d14fa1e4230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:39.868974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61c242-3b5a-43d9-880f-4d14fa1e4230 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:39.869250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61c242-3b5a-43d9-880f-4d14fa1e4230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:39.869518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61c242-3b5a-43d9-880f-4d14fa1e4230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:39.870190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61c242-3b5a-43d9-880f-4d14fa1e4230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:39.871770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61c242-3b5a-43d9-880f-4d14fa1e4230 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:39.882864 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1081/2177] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:39.887702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adf95c-0d33-441b-84f9-e8792afa03f9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:39.898329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adf95c-0d33-441b-84f9-e8792afa03f9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:39.903378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf89486-6a58-47e9-a60e-a6d083c68131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:39.904019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf89486-6a58-47e9-a60e-a6d083c68131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:39.904321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf89486-6a58-47e9-a60e-a6d083c68131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:39.904573 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf89486-6a58-47e9-a60e-a6d083c68131 admin admin] PATH_INFO: `/v3/projects/d843d825ffba4943902fcf3c24f6409b/users/09279d3676dc48ae8ffa33928a38220b/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:39.921066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf89486-6a58-47e9-a60e-a6d083c68131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d843d825ffba4943902fcf3c24f6409b, user_id=09279d3676dc48ae8ffa33928a38220b, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:39.923838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf89486-6a58-47e9-a60e-a6d083c68131 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:39.929487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:39.998786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1097/2178] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:39 2022] PUT /identity/v3/projects/d843d825ffba4943902fcf3c24f6409b/users/09279d3676dc48ae8ffa33928a38220b/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:50:40.012694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e1a7a-0f4d-496f-a7c5-79c4e9e64640 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:40.012979 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e1a7a-0f4d-496f-a7c5-79c4e9e64640 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:40.013246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e1a7a-0f4d-496f-a7c5-79c4e9e64640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:40.042957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-cb9e1a7a-0f4d-496f-a7c5-79c4e9e64640 None None] MFA Rules not processed for user `09279d3676dc48ae8ffa33928a38220b`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:40.044178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-cb9e1a7a-0f4d-496f-a7c5-79c4e9e64640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:40.111461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1082/2179] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:50:40.122968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf89486-6a58-47e9-a60e-a6d083c68131 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:40.128702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-abf89486-6a58-47e9-a60e-a6d083c68131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:40.183337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf89486-6a58-47e9-a60e-a6d083c68131 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:40.187173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda54dcb-14e3-41f7-aa7f-5992ce2233bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:40.187785 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda54dcb-14e3-41f7-aa7f-5992ce2233bd service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:40.188089 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda54dcb-14e3-41f7-aa7f-5992ce2233bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:40.188368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda54dcb-14e3-41f7-aa7f-5992ce2233bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:40.202263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda54dcb-14e3-41f7-aa7f-5992ce2233bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:40.204513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda54dcb-14e3-41f7-aa7f-5992ce2233bd service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:40.228437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1098/2180] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:40.239548 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e1a7a-0f4d-496f-a7c5-79c4e9e64640 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:40.320121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e1a7a-0f4d-496f-a7c5-79c4e9e64640 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:40.325090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:40.325875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:40.326254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:40.326634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] PATH_INFO: `/v3/users/09279d3676dc48ae8ffa33928a38220b` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:40.327876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09279d3676dc48ae8ffa33928a38220b)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:40.330405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:40.343687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:40.550746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09279d3676dc48ae8ffa33928a38220b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:40.634476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:40.665895 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09279d3676dc48ae8ffa33928a38220b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:40.666567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09279d3676dc48ae8ffa33928a38220b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:40.674214 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09279d3676dc48ae8ffa33928a38220b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:40.687433 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09279d3676dc48ae8ffa33928a38220b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:40.693920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1083/2181] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:40 2022] DELETE /identity/v3/users/09279d3676dc48ae8ffa33928a38220b => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:40.698340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed48f57-76e8-4cd1-be3b-1ffefee9d93f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:40.698624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed48f57-76e8-4cd1-be3b-1ffefee9d93f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:40.698901 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed48f57-76e8-4cd1-be3b-1ffefee9d93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:40.719787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9ed48f57-76e8-4cd1-be3b-1ffefee9d93f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:40.721155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9ed48f57-76e8-4cd1-be3b-1ffefee9d93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:40.784143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1099/2182] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:40.799252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:40.804121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:40.861841 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c5b59-67d8-474b-9118-3916c10563e8 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:40.866136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f4c9b-f90e-4490-819c-7944b7555114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:40.866775 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f4c9b-f90e-4490-819c-7944b7555114 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:40.867057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f4c9b-f90e-4490-819c-7944b7555114 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:40.867331 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f4c9b-f90e-4490-819c-7944b7555114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:40.876585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f4c9b-f90e-4490-819c-7944b7555114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:40.878104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f4c9b-f90e-4490-819c-7944b7555114 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:40.888626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1084/2183] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:41.290143 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:41.437824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed48f57-76e8-4cd1-be3b-1ffefee9d93f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:41.439279 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9ed48f57-76e8-4cd1-be3b-1ffefee9d93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:41.451617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed48f57-76e8-4cd1-be3b-1ffefee9d93f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:41.453919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:41.458324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:41.459906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:41.460187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:41.460458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] PATH_INFO: `/v3/projects/d843d825ffba4943902fcf3c24f6409b` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:41.461583 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d843d825ffba4943902fcf3c24f6409b)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:41.463839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:41.588858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd843d825ffba4943902fcf3c24f6409b', 'request_id': 'req-9fb4e358-8e9d-4d1a-affe-d10848a363c4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:41.604270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd843d825ffba4943902fcf3c24f6409b', 'request_id': 'req-9fb4e358-8e9d-4d1a-affe-d10848a363c4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:41.653969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd843d825ffba4943902fcf3c24f6409b', 'request_id': 'req-9fb4e358-8e9d-4d1a-affe-d10848a363c4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:41.666039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] The token cache is being invalidate because project d843d825ffba4943902fcf3c24f6409b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:41.666332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:41.667027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fb4e358-8e9d-4d1a-affe-d10848a363c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:41.668532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1100/2184] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:41 2022] DELETE /identity/v3/projects/d843d825ffba4943902fcf3c24f6409b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:41.673584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df7f7-4bb5-44eb-be82-ced5aafa13f4 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:41.674106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df7f7-4bb5-44eb-be82-ced5aafa13f4 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:41.674198 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df7f7-4bb5-44eb-be82-ced5aafa13f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:41.694404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4d9df7f7-4bb5-44eb-be82-ced5aafa13f4 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:41.699687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4d9df7f7-4bb5-44eb-be82-ced5aafa13f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:41.743971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:41.762774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1085/2185] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:41.770524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8a30c-80a4-41eb-9108-eb933883d766 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:41.770810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8a30c-80a4-41eb-9108-eb933883d766 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:41.771200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8a30c-80a4-41eb-9108-eb933883d766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:41.791938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-70c8a30c-80a4-41eb-9108-eb933883d766 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:41.793138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-70c8a30c-80a4-41eb-9108-eb933883d766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:41.801652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1101/2186] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:41.806083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9df7f7-4bb5-44eb-be82-ced5aafa13f4 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:41.808850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:41.815744 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9df7f7-4bb5-44eb-be82-ced5aafa13f4 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:41.820327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f102dd-08fa-41a8-8876-e740726fa7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:41.820916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f102dd-08fa-41a8-8876-e740726fa7df admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:41.821190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f102dd-08fa-41a8-8876-e740726fa7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:41.828972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f102dd-08fa-41a8-8876-e740726fa7df admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:41.828972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f102dd-08fa-41a8-8876-e740726fa7df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:50:41.828972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f102dd-08fa-41a8-8876-e740726fa7df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:41.828972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f102dd-08fa-41a8-8876-e740726fa7df admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:41.837499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1086/2187] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:50:41.840292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8a30c-80a4-41eb-9108-eb933883d766 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:41.851410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8a30c-80a4-41eb-9108-eb933883d766 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:41.860793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f12e18-851a-4f2f-a006-7cfee408c874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:41.860793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f12e18-851a-4f2f-a006-7cfee408c874 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:41.860793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f12e18-851a-4f2f-a006-7cfee408c874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:41.860793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f12e18-851a-4f2f-a006-7cfee408c874 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:41.861120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f12e18-851a-4f2f-a006-7cfee408c874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:41.861120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f12e18-851a-4f2f-a006-7cfee408c874 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:41.958202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1102/2188] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:41 2022] POST /identity/v3/projects => generated 369 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:41.963364 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f102dd-08fa-41a8-8876-e740726fa7df admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:41.973356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f102dd-08fa-41a8-8876-e740726fa7df admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:42.032451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cea58f-f938-4a7e-bd55-3d48c2904119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:42.033062 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cea58f-f938-4a7e-bd55-3d48c2904119 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:42.033340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cea58f-f938-4a7e-bd55-3d48c2904119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:42.033616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cea58f-f938-4a7e-bd55-3d48c2904119 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:42.034287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cea58f-f938-4a7e-bd55-3d48c2904119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:42.036204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cea58f-f938-4a7e-bd55-3d48c2904119 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:42.171517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1087/2189] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:41 2022] POST /identity/v3/users => generated 285 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:42.175494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f12e18-851a-4f2f-a006-7cfee408c874 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:42.186038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f12e18-851a-4f2f-a006-7cfee408c874 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:42.190525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a319-37bb-47e1-98d7-d1f283ac838a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:42.191110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a319-37bb-47e1-98d7-d1f283ac838a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:42.191390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a319-37bb-47e1-98d7-d1f283ac838a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:42.191659 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a319-37bb-47e1-98d7-d1f283ac838a admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:42.192290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4a319-37bb-47e1-98d7-d1f283ac838a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:42.194647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4a319-37bb-47e1-98d7-d1f283ac838a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:42.205202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1103/2190] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:42.209441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cea58f-f938-4a7e-bd55-3d48c2904119 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:42.219763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cea58f-f938-4a7e-bd55-3d48c2904119 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:42.224317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6873025-4077-4f56-b655-cb71518b2fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:42.226074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6873025-4077-4f56-b655-cb71518b2fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:42.226074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6873025-4077-4f56-b655-cb71518b2fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:42.226074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6873025-4077-4f56-b655-cb71518b2fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:42.226074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6873025-4077-4f56-b655-cb71518b2fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:42.227807 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6873025-4077-4f56-b655-cb71518b2fe5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:42.238929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1088/2191] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:42.243693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a319-37bb-47e1-98d7-d1f283ac838a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:42.253530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a319-37bb-47e1-98d7-d1f283ac838a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:42.259026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fa081-ad09-4b59-a29a-399ca3bb0a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:42.259663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fa081-ad09-4b59-a29a-399ca3bb0a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:42.259936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fa081-ad09-4b59-a29a-399ca3bb0a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:42.260206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fa081-ad09-4b59-a29a-399ca3bb0a52 admin admin] PATH_INFO: `/v3/projects/d1f09cb3ec0147b39302fd2df9e89a5b/users/29375fdec6fd4a93ab440a11d8aa6fc4/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:42.276109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fa081-ad09-4b59-a29a-399ca3bb0a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f09cb3ec0147b39302fd2df9e89a5b, user_id=29375fdec6fd4a93ab440a11d8aa6fc4, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:42.278799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fa081-ad09-4b59-a29a-399ca3bb0a52 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:42.283420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:42.372644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:42.451425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1104/2192] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:42 2022] PUT /identity/v3/projects/d1f09cb3ec0147b39302fd2df9e89a5b/users/29375fdec6fd4a93ab440a11d8aa6fc4/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:42.462556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c2ff5-d201-4ad0-a71a-933f12e568dd None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:42.462843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c2ff5-d201-4ad0-a71a-933f12e568dd None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:42.463115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c2ff5-d201-4ad0-a71a-933f12e568dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:42.492374 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a04c2ff5-d201-4ad0-a71a-933f12e568dd None None] MFA Rules not processed for user `29375fdec6fd4a93ab440a11d8aa6fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:42.493636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a04c2ff5-d201-4ad0-a71a-933f12e568dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:42.557390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1089/2193] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:42.569607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fa081-ad09-4b59-a29a-399ca3bb0a52 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:42.571387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6a1fa081-ad09-4b59-a29a-399ca3bb0a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:42.618554 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:42.634522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:42.653936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fa081-ad09-4b59-a29a-399ca3bb0a52 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:42.657754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf5a37-2c5a-435d-aac1-b842c5e044d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab54ee52004746eaadb5c2970e59afc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:42.658357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf5a37-2c5a-435d-aac1-b842c5e044d9 service glance] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:42.658639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf5a37-2c5a-435d-aac1-b842c5e044d9 service glance] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:42.658921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf5a37-2c5a-435d-aac1-b842c5e044d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:42.668005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf5a37-2c5a-435d-aac1-b842c5e044d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:42.669436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf5a37-2c5a-435d-aac1-b842c5e044d9 service glance] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:42.679123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1105/2194] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:50:42.856929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:42.886377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:43.666448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:43.788873 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:44.407660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b59f6-1635-4b95-8a86-456aa4778775 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:44.408215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b59f6-1635-4b95-8a86-456aa4778775 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:44.408215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b59f6-1635-4b95-8a86-456aa4778775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:44.430125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ce9b59f6-1635-4b95-8a86-456aa4778775 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:44.431820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ce9b59f6-1635-4b95-8a86-456aa4778775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:44.504080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1090/2195] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:50:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:44.517596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf5a37-2c5a-435d-aac1-b842c5e044d9 service glance] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:44.519776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-96bf5a37-2c5a-435d-aac1-b842c5e044d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:44.571053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b59f6-1635-4b95-8a86-456aa4778775 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:44.572549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ce9b59f6-1635-4b95-8a86-456aa4778775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:44.617941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf5a37-2c5a-435d-aac1-b842c5e044d9 service glance] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:44.624156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:44.624926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:44.627461 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:44.627461 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] PATH_INFO: `/v3/users/b36a1afe219d44f8ac5a4b11d4f87b71` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:44.628744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36a1afe219d44f8ac5a4b11d4f87b71)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:44.632199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:44.662981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b59f6-1635-4b95-8a86-456aa4778775 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:44.670855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eeca13-2b5d-4f59-b6d8-374992b30feb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:44.672472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eeca13-2b5d-4f59-b6d8-374992b30feb service swift] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:44.672886 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eeca13-2b5d-4f59-b6d8-374992b30feb service swift] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:44.673233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eeca13-2b5d-4f59-b6d8-374992b30feb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:44.687613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eeca13-2b5d-4f59-b6d8-374992b30feb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:44.689738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eeca13-2b5d-4f59-b6d8-374992b30feb service swift] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:44.710579 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1091/2196] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:50:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 03:50:44.929873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36a1afe219d44f8ac5a4b11d4f87b71'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:44.960136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36a1afe219d44f8ac5a4b11d4f87b71'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:45.081112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36a1afe219d44f8ac5a4b11d4f87b71'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:45.084440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36a1afe219d44f8ac5a4b11d4f87b71'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:45.128789 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36a1afe219d44f8ac5a4b11d4f87b71'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:45.157262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1106/2197] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:44 2022] DELETE /identity/v3/users/b36a1afe219d44f8ac5a4b11d4f87b71 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:45.171836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc23223-f917-491c-b988-c8464ce0b3e8 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:45.172354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc23223-f917-491c-b988-c8464ce0b3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:45.172688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc23223-f917-491c-b988-c8464ce0b3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:45.218565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-5dc23223-f917-491c-b988-c8464ce0b3e8 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:45.224009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5dc23223-f917-491c-b988-c8464ce0b3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:45.340503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:45.395081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1092/2198] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:45.424304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:45.433786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:45.648717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc1a7c-2dc7-4bc5-b6a0-6fea3fe10a4d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:45.648717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169844a-87f3-4325-a8a3-95d39a95be47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:45.648717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169844a-87f3-4325-a8a3-95d39a95be47 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:45.648717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169844a-87f3-4325-a8a3-95d39a95be47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:45.659900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169844a-87f3-4325-a8a3-95d39a95be47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:45.659900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169844a-87f3-4325-a8a3-95d39a95be47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:45.659900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169844a-87f3-4325-a8a3-95d39a95be47 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:45.659900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1107/2199] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:45.847025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:46.257920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc23223-f917-491c-b988-c8464ce0b3e8 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:46.259388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5dc23223-f917-491c-b988-c8464ce0b3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:46.287662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc23223-f917-491c-b988-c8464ce0b3e8 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:46.306194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:46.310291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:46.310291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:46.310291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] PATH_INFO: `/v3/projects/5ae68e0d3ded46808ca91e5c35fa786e` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:46.315554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ae68e0d3ded46808ca91e5c35fa786e)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:46.317551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:46.560221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ae68e0d3ded46808ca91e5c35fa786e', 'request_id': 'req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:46.643361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ae68e0d3ded46808ca91e5c35fa786e', 'request_id': 'req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:46.659563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ae68e0d3ded46808ca91e5c35fa786e', 'request_id': 'req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:46.676451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:46.696403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] The token cache is being invalidate because project 5ae68e0d3ded46808ca91e5c35fa786e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:46.696820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:46.702753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6df8d95d-e003-4d12-9d3f-8ac2c3e3174e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:46.708609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1093/2200] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:46 2022] DELETE /identity/v3/projects/5ae68e0d3ded46808ca91e5c35fa786e => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:46.712952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169844a-87f3-4325-a8a3-95d39a95be47 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:46.735692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169844a-87f3-4325-a8a3-95d39a95be47 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:46.817172 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3c5dd-2841-4cc7-bf3f-a196f29107c9 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:46.817525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3c5dd-2841-4cc7-bf3f-a196f29107c9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:46.818625 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3c5dd-2841-4cc7-bf3f-a196f29107c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:46.845948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-8fd3c5dd-2841-4cc7-bf3f-a196f29107c9 None None] MFA Rules not processed for user `fafa609d94334db6907b401e66cb3448`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:46.847404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8fd3c5dd-2841-4cc7-bf3f-a196f29107c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:46.856040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:46.856774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:46.857119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:46.857490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] PATH_INFO: `/v3/users/0760dc6091764d2b99ed5a3cc1018cd6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:46.858989 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0760dc6091764d2b99ed5a3cc1018cd6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:46.860782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:46.915298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1094/2201] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:47.152634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0760dc6091764d2b99ed5a3cc1018cd6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:47.168284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0760dc6091764d2b99ed5a3cc1018cd6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:47.260087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0760dc6091764d2b99ed5a3cc1018cd6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:47.260776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0760dc6091764d2b99ed5a3cc1018cd6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:47.269092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0760dc6091764d2b99ed5a3cc1018cd6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:47.277146 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1108/2202] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:46 2022] DELETE /identity/v3/users/0760dc6091764d2b99ed5a3cc1018cd6 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:47.793943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3c5dd-2841-4cc7-bf3f-a196f29107c9 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:47.795402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8fd3c5dd-2841-4cc7-bf3f-a196f29107c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:47.814471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3c5dd-2841-4cc7-bf3f-a196f29107c9 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:47.829419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:47.930808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:47.930808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:47.930808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:47.930808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] PATH_INFO: `/v3/projects/c555855139714fdfaed60aaa0f11a563` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:47.932096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c555855139714fdfaed60aaa0f11a563)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:47.932096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:48.132610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c555855139714fdfaed60aaa0f11a563', 'request_id': 'req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:48.228354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c555855139714fdfaed60aaa0f11a563', 'request_id': 'req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:48.234922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c555855139714fdfaed60aaa0f11a563', 'request_id': 'req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:48.247146 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] The token cache is being invalidate because project c555855139714fdfaed60aaa0f11a563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:48.247453 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:48.248119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:48.249661 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1095/2203] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:47 2022] DELETE /identity/v3/projects/c555855139714fdfaed60aaa0f11a563 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:48.253879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:48.255414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:48.269559 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa2846-0a8a-441b-afda-0a1d66900687 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:48.377872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:48.378798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:48.379161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:48.379545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] PATH_INFO: `/v3/users/be6cbb0ca30a4704af55da61381b7051` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:48.382409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] RBAC: Authorizing `identity:delete_user(user_id=be6cbb0ca30a4704af55da61381b7051)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:48.384662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:48.466469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:48.691106 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be6cbb0ca30a4704af55da61381b7051'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:48.709535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be6cbb0ca30a4704af55da61381b7051'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:48.867738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be6cbb0ca30a4704af55da61381b7051'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:48.868395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be6cbb0ca30a4704af55da61381b7051'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:48.892571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7a9dc62b-a5d1-4b1c-815d-d124e709992f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be6cbb0ca30a4704af55da61381b7051'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:48.907304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1109/2204] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:48 2022] DELETE /identity/v3/users/be6cbb0ca30a4704af55da61381b7051 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:49.274507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:49.524591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:49.526115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:49.540744 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97f185-8ca3-48a6-b9c6-f9c2822258ce admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:49.598808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:49.599984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:49.600303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:49.600660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] PATH_INFO: `/v3/projects/981b5a46bff2436683d2fdf97754e062` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:49.601909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=981b5a46bff2436683d2fdf97754e062)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:49.603954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:49.815277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '981b5a46bff2436683d2fdf97754e062', 'request_id': 'req-0a2e1d58-85aa-426c-8196-05ac82d69fbf'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:49.923080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '981b5a46bff2436683d2fdf97754e062', 'request_id': 'req-0a2e1d58-85aa-426c-8196-05ac82d69fbf'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:49.930595 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '981b5a46bff2436683d2fdf97754e062', 'request_id': 'req-0a2e1d58-85aa-426c-8196-05ac82d69fbf'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:49.951753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] The token cache is being invalidate because project 981b5a46bff2436683d2fdf97754e062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:50:49.952151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:49.952707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0a2e1d58-85aa-426c-8196-05ac82d69fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:49.954210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1096/2205] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:50:49 2022] DELETE /identity/v3/projects/981b5a46bff2436683d2fdf97754e062 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:49.959309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3736aa7-9859-4749-ad94-89f6f4955f30 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:49.960435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3736aa7-9859-4749-ad94-89f6f4955f30 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:49.960723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3736aa7-9859-4749-ad94-89f6f4955f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:49.985485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c3736aa7-9859-4749-ad94-89f6f4955f30 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:49.986368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c3736aa7-9859-4749-ad94-89f6f4955f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:50.062356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1110/2206] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:50.070666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cecf71-133f-4f53-8579-d0f3f1e70f2c None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:50.070994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cecf71-133f-4f53-8579-d0f3f1e70f2c None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:50.071257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cecf71-133f-4f53-8579-d0f3f1e70f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:50.103328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-35cecf71-133f-4f53-8579-d0f3f1e70f2c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:50.104556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-35cecf71-133f-4f53-8579-d0f3f1e70f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:50.122376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1097/2207] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:50.126992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3736aa7-9859-4749-ad94-89f6f4955f30 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:50.139279 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3736aa7-9859-4749-ad94-89f6f4955f30 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:50.144488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03921d93-fc36-4005-a5f8-c59f99cb4a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:50.145345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03921d93-fc36-4005-a5f8-c59f99cb4a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:50.145849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03921d93-fc36-4005-a5f8-c59f99cb4a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:50.146962 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03921d93-fc36-4005-a5f8-c59f99cb4a70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:50.147639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03921d93-fc36-4005-a5f8-c59f99cb4a70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:50:50.147959 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03921d93-fc36-4005-a5f8-c59f99cb4a70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:50.149950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03921d93-fc36-4005-a5f8-c59f99cb4a70 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:50.167192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1111/2208] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:50:50.174985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cecf71-133f-4f53-8579-d0f3f1e70f2c None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:50.192543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cecf71-133f-4f53-8579-d0f3f1e70f2c None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:50.200889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88053b-24c0-4140-acac-9ce5b86ece0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:50.202024 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88053b-24c0-4140-acac-9ce5b86ece0c admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:50.202304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88053b-24c0-4140-acac-9ce5b86ece0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:50.202616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88053b-24c0-4140-acac-9ce5b86ece0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:50.203426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88053b-24c0-4140-acac-9ce5b86ece0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:50.205447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88053b-24c0-4140-acac-9ce5b86ece0c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:50.311581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1098/2209] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:50:50 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:50.313449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03921d93-fc36-4005-a5f8-c59f99cb4a70 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:50.326416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03921d93-fc36-4005-a5f8-c59f99cb4a70 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:50.400619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:50.433149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a866861-3b8d-4aa0-b20e-1804e466a8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:50.438316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a866861-3b8d-4aa0-b20e-1804e466a8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:50.438316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a866861-3b8d-4aa0-b20e-1804e466a8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:50.438316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a866861-3b8d-4aa0-b20e-1804e466a8d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:50.438316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a866861-3b8d-4aa0-b20e-1804e466a8d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:50.438316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a866861-3b8d-4aa0-b20e-1804e466a8d3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:50.507120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:50.548704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1112/2210] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:50:50 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:50:50.550376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88053b-24c0-4140-acac-9ce5b86ece0c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:50.564061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88053b-24c0-4140-acac-9ce5b86ece0c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:50.593623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dad89-3459-4f7c-8a08-978e48d34dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:50.596413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dad89-3459-4f7c-8a08-978e48d34dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:50.596643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dad89-3459-4f7c-8a08-978e48d34dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:50.596949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dad89-3459-4f7c-8a08-978e48d34dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:50.598038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dad89-3459-4f7c-8a08-978e48d34dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:50.602430 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dad89-3459-4f7c-8a08-978e48d34dcb admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:50.636334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1099/2211] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:50.640463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a866861-3b8d-4aa0-b20e-1804e466a8d3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:50.652547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a866861-3b8d-4aa0-b20e-1804e466a8d3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:50.669395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b45fd-cac6-4a6d-bdc5-9187a26fdfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:50.670906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b45fd-cac6-4a6d-bdc5-9187a26fdfee admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:50.670906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b45fd-cac6-4a6d-bdc5-9187a26fdfee admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:50.670906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b45fd-cac6-4a6d-bdc5-9187a26fdfee admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:50.673690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b45fd-cac6-4a6d-bdc5-9187a26fdfee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:50.676592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b45fd-cac6-4a6d-bdc5-9187a26fdfee admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:50.698978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1113/2212] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:50:50.703237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dad89-3459-4f7c-8a08-978e48d34dcb admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:50.715278 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dad89-3459-4f7c-8a08-978e48d34dcb admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:50.720550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42828624-9502-4780-aec8-6048e328198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:50.721340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828624-9502-4780-aec8-6048e328198b admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:50.721871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828624-9502-4780-aec8-6048e328198b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:50.728322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828624-9502-4780-aec8-6048e328198b admin admin] PATH_INFO: `/v3/projects/0fd9cfe6621e4679a180dd73164123a8/users/824096843a544a0a8eb3120b662a9086/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:50.740702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42828624-9502-4780-aec8-6048e328198b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd9cfe6621e4679a180dd73164123a8, user_id=824096843a544a0a8eb3120b662a9086, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:50.743585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42828624-9502-4780-aec8-6048e328198b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:50.855292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1100/2213] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:50:50 2022] PUT /identity/v3/projects/0fd9cfe6621e4679a180dd73164123a8/users/824096843a544a0a8eb3120b662a9086/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:50.928828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca2122-1536-4236-8357-bb6383eb497b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:50.929471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca2122-1536-4236-8357-bb6383eb497b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:50.930025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca2122-1536-4236-8357-bb6383eb497b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:50.974253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f7ca2122-1536-4236-8357-bb6383eb497b None None] MFA Rules not processed for user `824096843a544a0a8eb3120b662a9086`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:50.975436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f7ca2122-1536-4236-8357-bb6383eb497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:51.055881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:51.070187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1114/2214] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:51.083299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42828624-9502-4780-aec8-6048e328198b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:51.085199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-42828624-9502-4780-aec8-6048e328198b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:51.174767 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42828624-9502-4780-aec8-6048e328198b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:51.186105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c579bc-dafc-4eea-a47e-538c43896287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:51.186105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c579bc-dafc-4eea-a47e-538c43896287 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:51.186105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c579bc-dafc-4eea-a47e-538c43896287 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:51.186105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c579bc-dafc-4eea-a47e-538c43896287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:51.198932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c579bc-dafc-4eea-a47e-538c43896287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:51.198932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c579bc-dafc-4eea-a47e-538c43896287 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:51.207822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1101/2215] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:50:51.588686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:51.811028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:51.957553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:52.457051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:52.554153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:52.803226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:52.871487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:52.943905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:53.083075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:53.599299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:53.810792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:54.825445 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:55.114626 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:55.431442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:55.467077 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:56.122509 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:56.305749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:56.634566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:56.677114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:57.417430 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:50:58.587945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca2122-1536-4236-8357-bb6383eb497b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:58.596779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f7ca2122-1536-4236-8357-bb6383eb497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:58.687162 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca2122-1536-4236-8357-bb6383eb497b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:58.692958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:58.695254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:58.695616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:58.696164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] PATH_INFO: `/v3/users/2a01865a4cc34624ade14242ee521a08` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:58.697782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a01865a4cc34624ade14242ee521a08)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:58.700837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:58.925653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a01865a4cc34624ade14242ee521a08'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:58.970064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a01865a4cc34624ade14242ee521a08'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:59.049397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a01865a4cc34624ade14242ee521a08'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:59.050393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a01865a4cc34624ade14242ee521a08'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:59.064693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a01865a4cc34624ade14242ee521a08'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:50:59.074353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1115/2216] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:50:58 2022] DELETE /identity/v3/users/2a01865a4cc34624ade14242ee521a08 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:50:59.078943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1503d-d099-4a13-ac3b-6b011a4c8ee2 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:59.079916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1503d-d099-4a13-ac3b-6b011a4c8ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:59.080569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1503d-d099-4a13-ac3b-6b011a4c8ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:59.118345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b4c1503d-d099-4a13-ac3b-6b011a4c8ee2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:50:59.120105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b4c1503d-d099-4a13-ac3b-6b011a4c8ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:59.214441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1102/2217] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:50:59.220178 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:50:59.221849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:50:59.386124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf24d7-07ad-48c4-a144-d86f18927fe3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:50:59.401842 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc8129-3af1-4401-aafe-6803ebbbc7be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:50:59.402865 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc8129-3af1-4401-aafe-6803ebbbc7be service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:50:59.403317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc8129-3af1-4401-aafe-6803ebbbc7be service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:50:59.403977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc8129-3af1-4401-aafe-6803ebbbc7be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:50:59.417326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc8129-3af1-4401-aafe-6803ebbbc7be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:50:59.418977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc8129-3af1-4401-aafe-6803ebbbc7be service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:50:59.445589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1116/2218] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:50:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:00.002876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1503d-d099-4a13-ac3b-6b011a4c8ee2 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:00.005889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b4c1503d-d099-4a13-ac3b-6b011a4c8ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:00.056274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1503d-d099-4a13-ac3b-6b011a4c8ee2 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:00.064064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:00.064510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:00.064880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:00.065255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] PATH_INFO: `/v3/projects/0e0f154b85d043b39eddbc92f7642c30` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:00.066627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e0f154b85d043b39eddbc92f7642c30)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:00.070310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:00.082383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:00.235991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e0f154b85d043b39eddbc92f7642c30', 'request_id': 'req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:00.322240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e0f154b85d043b39eddbc92f7642c30', 'request_id': 'req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:00.332538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e0f154b85d043b39eddbc92f7642c30', 'request_id': 'req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:00.359751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] The token cache is being invalidate because project 0e0f154b85d043b39eddbc92f7642c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:00.360217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:00.360982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:00.368598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc8129-3af1-4401-aafe-6803ebbbc7be service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:00.369147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1103/2219] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:00 2022] DELETE /identity/v3/projects/0e0f154b85d043b39eddbc92f7642c30 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:00.382526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc8129-3af1-4401-aafe-6803ebbbc7be service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:00.392453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:00.551087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:00.553176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:00.554251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:00.554561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] PATH_INFO: `/v3/users/044c3ecc3b4b41c4a5dfa6dd18c327c0` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:00.557114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=044c3ecc3b4b41c4a5dfa6dd18c327c0)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:00.573318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:00.858310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044c3ecc3b4b41c4a5dfa6dd18c327c0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:00.873606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044c3ecc3b4b41c4a5dfa6dd18c327c0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:00.947943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044c3ecc3b4b41c4a5dfa6dd18c327c0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:00.948675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044c3ecc3b4b41c4a5dfa6dd18c327c0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:00.958132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6d1675aa-7d47-471e-8818-26a9f7a256c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044c3ecc3b4b41c4a5dfa6dd18c327c0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:00.969065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1117/2220] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:00 2022] DELETE /identity/v3/users/044c3ecc3b4b41c4a5dfa6dd18c327c0 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:01.414385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:01.419413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:01.435447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcc4e2-b703-497d-a0fe-f447f26fbcf5 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:01.504761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:01.505555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:01.506545 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:01.506821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] PATH_INFO: `/v3/projects/01a0694f564d4450a0b5808cade74259` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:01.508436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01a0694f564d4450a0b5808cade74259)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:01.510765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:01.638175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01a0694f564d4450a0b5808cade74259', 'request_id': 'req-443d2b99-bd48-4bb8-986b-af9673660375'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:01.704853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01a0694f564d4450a0b5808cade74259', 'request_id': 'req-443d2b99-bd48-4bb8-986b-af9673660375'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:01.714607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01a0694f564d4450a0b5808cade74259', 'request_id': 'req-443d2b99-bd48-4bb8-986b-af9673660375'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:01.742786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] The token cache is being invalidate because project 01a0694f564d4450a0b5808cade74259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:01.743125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:01.744392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-443d2b99-bd48-4bb8-986b-af9673660375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:01.749776 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1104/2221] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:01 2022] DELETE /identity/v3/projects/01a0694f564d4450a0b5808cade74259 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:01.752800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070711d-ced3-4722-b6ff-fd735c050cbc None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:01.753279 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070711d-ced3-4722-b6ff-fd735c050cbc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:01.753374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070711d-ced3-4722-b6ff-fd735c050cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:01.782374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8070711d-ced3-4722-b6ff-fd735c050cbc None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:01.783792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8070711d-ced3-4722-b6ff-fd735c050cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:01.905606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1118/2222] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:01.907325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9a43-6256-4be8-80f8-e08eeece4527 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:01.907637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9a43-6256-4be8-80f8-e08eeece4527 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:01.907915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9a43-6256-4be8-80f8-e08eeece4527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:01.948472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a6fe9a43-6256-4be8-80f8-e08eeece4527 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:01.949653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a6fe9a43-6256-4be8-80f8-e08eeece4527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:01.969766 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1105/2223] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:01.972362 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070711d-ced3-4722-b6ff-fd735c050cbc None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:01.993697 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070711d-ced3-4722-b6ff-fd735c050cbc None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:02.001797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa84b08-546f-4bee-9dbd-216b72c6f324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:02.002466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa84b08-546f-4bee-9dbd-216b72c6f324 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:02.002829 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa84b08-546f-4bee-9dbd-216b72c6f324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:02.003636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa84b08-546f-4bee-9dbd-216b72c6f324 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:02.004311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa84b08-546f-4bee-9dbd-216b72c6f324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:51:02.006366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa84b08-546f-4bee-9dbd-216b72c6f324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:02.008403 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa84b08-546f-4bee-9dbd-216b72c6f324 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:02.027151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1119/2224] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:51:02.031343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9a43-6256-4be8-80f8-e08eeece4527 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:02.041872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9a43-6256-4be8-80f8-e08eeece4527 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:02.048649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98356e9-b38c-427e-bb79-73ccef4ccce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:02.049250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98356e9-b38c-427e-bb79-73ccef4ccce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:02.049525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98356e9-b38c-427e-bb79-73ccef4ccce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:02.049836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98356e9-b38c-427e-bb79-73ccef4ccce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:02.050717 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98356e9-b38c-427e-bb79-73ccef4ccce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:02.052668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98356e9-b38c-427e-bb79-73ccef4ccce3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:02.073443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:02.173572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1106/2225] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:02 2022] POST /identity/v3/projects => generated 401 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:02.174753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa84b08-546f-4bee-9dbd-216b72c6f324 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:02.188717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa84b08-546f-4bee-9dbd-216b72c6f324 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:02.212049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:02.294629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6075af-266b-4c94-abe9-c23185733108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:02.294629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6075af-266b-4c94-abe9-c23185733108 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:02.294629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6075af-266b-4c94-abe9-c23185733108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:02.294629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6075af-266b-4c94-abe9-c23185733108 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:02.295105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6075af-266b-4c94-abe9-c23185733108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:02.295105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6075af-266b-4c94-abe9-c23185733108 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:02.440817 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1120/2226] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:02 2022] POST /identity/v3/users => generated 301 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:51:02.441986 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98356e9-b38c-427e-bb79-73ccef4ccce3 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:02.453738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98356e9-b38c-427e-bb79-73ccef4ccce3 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:02.487550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55a3ff-8106-407f-abef-b84e408b1f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:02.488187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55a3ff-8106-407f-abef-b84e408b1f8e admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:02.488463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55a3ff-8106-407f-abef-b84e408b1f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:02.488733 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55a3ff-8106-407f-abef-b84e408b1f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:02.489422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55a3ff-8106-407f-abef-b84e408b1f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:02.491664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55a3ff-8106-407f-abef-b84e408b1f8e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:02.506718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1107/2227] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:51:02.512215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6075af-266b-4c94-abe9-c23185733108 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:02.523175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6075af-266b-4c94-abe9-c23185733108 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:02.527877 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a0cbb-d588-407d-aaaa-6b559518ff77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:02.528480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a0cbb-d588-407d-aaaa-6b559518ff77 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:02.528787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a0cbb-d588-407d-aaaa-6b559518ff77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:02.529044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a0cbb-d588-407d-aaaa-6b559518ff77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:02.529757 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a0cbb-d588-407d-aaaa-6b559518ff77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:02.531381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a0cbb-d588-407d-aaaa-6b559518ff77 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:02.542646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1121/2228] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:02.546804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55a3ff-8106-407f-abef-b84e408b1f8e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:02.557388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55a3ff-8106-407f-abef-b84e408b1f8e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:02.562059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec58dc-9601-4e22-b50f-8c91c0229226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:02.562680 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec58dc-9601-4e22-b50f-8c91c0229226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:02.562962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec58dc-9601-4e22-b50f-8c91c0229226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:02.563243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec58dc-9601-4e22-b50f-8c91c0229226 admin admin] PATH_INFO: `/v3/projects/0e3254e44c8f435b9fbfc33bf5e12381/users/982649dd80c340c8b00b5d795504f9b8/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:02.579075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec58dc-9601-4e22-b50f-8c91c0229226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3254e44c8f435b9fbfc33bf5e12381, user_id=982649dd80c340c8b00b5d795504f9b8, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:02.581863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec58dc-9601-4e22-b50f-8c91c0229226 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:02.692243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1108/2229] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:02 2022] PUT /identity/v3/projects/0e3254e44c8f435b9fbfc33bf5e12381/users/982649dd80c340c8b00b5d795504f9b8/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:02.704016 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd9b9c-2876-4d4f-a466-e8eb49a93c53 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:02.704274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd9b9c-2876-4d4f-a466-e8eb49a93c53 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:02.704650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd9b9c-2876-4d4f-a466-e8eb49a93c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:02.737354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-93fd9b9c-2876-4d4f-a466-e8eb49a93c53 None None] MFA Rules not processed for user `982649dd80c340c8b00b5d795504f9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:02.738619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-93fd9b9c-2876-4d4f-a466-e8eb49a93c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:02.820935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1122/2230] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:02 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:02.825941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec58dc-9601-4e22-b50f-8c91c0229226 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:02.914323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec58dc-9601-4e22-b50f-8c91c0229226 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:02.920973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b83cab-5f85-49da-9419-256feecf2754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:02.921613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b83cab-5f85-49da-9419-256feecf2754 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:02.921951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b83cab-5f85-49da-9419-256feecf2754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:02.922224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b83cab-5f85-49da-9419-256feecf2754 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:02.923065 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b83cab-5f85-49da-9419-256feecf2754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:02.925067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b83cab-5f85-49da-9419-256feecf2754 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:03.062937 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1109/2231] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:02 2022] POST /identity/v3/projects => generated 401 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:03.066644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd9b9c-2876-4d4f-a466-e8eb49a93c53 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:03.082530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd9b9c-2876-4d4f-a466-e8eb49a93c53 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:03.165622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f1f73-2aa8-4ccc-90e7-c957ad848d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:03.166383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1f73-2aa8-4ccc-90e7-c957ad848d36 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:03.166796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1f73-2aa8-4ccc-90e7-c957ad848d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:03.167150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1f73-2aa8-4ccc-90e7-c957ad848d36 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:03.168082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f1f73-2aa8-4ccc-90e7-c957ad848d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:03.170499 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f1f73-2aa8-4ccc-90e7-c957ad848d36 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:03.320422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1123/2232] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:03 2022] POST /identity/v3/users => generated 307 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:03.324564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b83cab-5f85-49da-9419-256feecf2754 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:03.338402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b83cab-5f85-49da-9419-256feecf2754 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:03.343527 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e525422-a0fc-4af8-a61f-2791bf9eae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:03.344140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e525422-a0fc-4af8-a61f-2791bf9eae21 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:03.344546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e525422-a0fc-4af8-a61f-2791bf9eae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:03.344843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e525422-a0fc-4af8-a61f-2791bf9eae21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:03.345554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e525422-a0fc-4af8-a61f-2791bf9eae21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:03.347399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e525422-a0fc-4af8-a61f-2791bf9eae21 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:03.372558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1110/2233] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:51:03.376893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f1f73-2aa8-4ccc-90e7-c957ad848d36 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:03.387879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f1f73-2aa8-4ccc-90e7-c957ad848d36 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:03.393155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d4276-9cb2-417f-8da6-42fe9400b143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:03.396868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d4276-9cb2-417f-8da6-42fe9400b143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:03.396868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d4276-9cb2-417f-8da6-42fe9400b143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:03.396868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d4276-9cb2-417f-8da6-42fe9400b143 admin admin] PATH_INFO: `/v3/projects/5ab0852151984ddb96cb1e0d8f510a4c/users/e4fbe98b7df04376a13ac8c6ce9fa4ce/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:03.442255 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:03.446993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d4276-9cb2-417f-8da6-42fe9400b143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab0852151984ddb96cb1e0d8f510a4c, user_id=e4fbe98b7df04376a13ac8c6ce9fa4ce, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:03.451385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d4276-9cb2-417f-8da6-42fe9400b143 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:03.539025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1124/2234] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:03 2022] PUT /identity/v3/projects/5ab0852151984ddb96cb1e0d8f510a4c/users/e4fbe98b7df04376a13ac8c6ce9fa4ce/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:51:03.543864 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e525422-a0fc-4af8-a61f-2791bf9eae21 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:03.557942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e525422-a0fc-4af8-a61f-2791bf9eae21 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:03.669382 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4f7c6-0588-4a91-baf7-0b19462a4f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:03.670120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f7c6-0588-4a91-baf7-0b19462a4f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:03.670994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f7c6-0588-4a91-baf7-0b19462a4f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:03.670994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f7c6-0588-4a91-baf7-0b19462a4f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:03.672290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4f7c6-0588-4a91-baf7-0b19462a4f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:03.674877 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4f7c6-0588-4a91-baf7-0b19462a4f5b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:03.701507 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1111/2235] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:03.702447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d4276-9cb2-417f-8da6-42fe9400b143 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:03.714289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d4276-9cb2-417f-8da6-42fe9400b143 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:03.719222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd7441-4339-4efd-8131-45ebbf3a93d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:03.719877 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7441-4339-4efd-8131-45ebbf3a93d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:03.720161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7441-4339-4efd-8131-45ebbf3a93d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:03.720527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7441-4339-4efd-8131-45ebbf3a93d6 admin admin] PATH_INFO: `/v3/projects/5ab0852151984ddb96cb1e0d8f510a4c/users/e4fbe98b7df04376a13ac8c6ce9fa4ce/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:03.722492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd7441-4339-4efd-8131-45ebbf3a93d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab0852151984ddb96cb1e0d8f510a4c, user_id=e4fbe98b7df04376a13ac8c6ce9fa4ce, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:03.725931 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd7441-4339-4efd-8131-45ebbf3a93d6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:03.744133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1125/2236] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:03 2022] PUT /identity/v3/projects/5ab0852151984ddb96cb1e0d8f510a4c/users/e4fbe98b7df04376a13ac8c6ce9fa4ce/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:51:03.755381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4359c-c8da-4c20-a502-e2a0ec65bc38 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:03.756234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4359c-c8da-4c20-a502-e2a0ec65bc38 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:03.756686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4359c-c8da-4c20-a502-e2a0ec65bc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:03.787693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-09d4359c-c8da-4c20-a502-e2a0ec65bc38 None None] MFA Rules not processed for user `e4fbe98b7df04376a13ac8c6ce9fa4ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:03.788934 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-09d4359c-c8da-4c20-a502-e2a0ec65bc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:03.836125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:03.927091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1112/2237] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:03 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:03.939799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd7441-4339-4efd-8131-45ebbf3a93d6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:03.942200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6fcd7441-4339-4efd-8131-45ebbf3a93d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:04.040848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd7441-4339-4efd-8131-45ebbf3a93d6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:04.047337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9feca22-7d85-4c10-a786-ebb0e95ae6cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:04.057286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feca22-7d85-4c10-a786-ebb0e95ae6cd service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:04.057286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feca22-7d85-4c10-a786-ebb0e95ae6cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:04.057286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feca22-7d85-4c10-a786-ebb0e95ae6cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:04.070219 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9feca22-7d85-4c10-a786-ebb0e95ae6cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:04.071968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9feca22-7d85-4c10-a786-ebb0e95ae6cd service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:04.085041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1126/2238] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:03 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:04.462656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4359c-c8da-4c20-a502-e2a0ec65bc38 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:04.474243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4359c-c8da-4c20-a502-e2a0ec65bc38 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:04.485178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbf82a-4041-456f-bf5c-e68967841ae0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:04.486654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbf82a-4041-456f-bf5c-e68967841ae0 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:04.487833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbf82a-4041-456f-bf5c-e68967841ae0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:04.488118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbf82a-4041-456f-bf5c-e68967841ae0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:04.718309 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbf82a-4041-456f-bf5c-e68967841ae0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:04.719850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbf82a-4041-456f-bf5c-e68967841ae0 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:04.743822 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1113/2239] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:04 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:04.747329 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9feca22-7d85-4c10-a786-ebb0e95ae6cd service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:04.752440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c9feca22-7d85-4c10-a786-ebb0e95ae6cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:04.847124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a4414-06f2-41ee-afd6-288669e544cc None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:04.847509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a4414-06f2-41ee-afd6-288669e544cc None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:04.847896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a4414-06f2-41ee-afd6-288669e544cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:04.916848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b24a4414-06f2-41ee-afd6-288669e544cc None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:04.921775 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b24a4414-06f2-41ee-afd6-288669e544cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:04.992455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9feca22-7d85-4c10-a786-ebb0e95ae6cd service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:05.003305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:05.022092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:05.023490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:05.024168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] PATH_INFO: `/v3/users/824096843a544a0a8eb3120b662a9086` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:05.026481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] RBAC: Authorizing `identity:delete_user(user_id=824096843a544a0a8eb3120b662a9086)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:05.033808 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:05.041058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1114/2240] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:51:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:05.100032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a4414-06f2-41ee-afd6-288669e544cc None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:05.101468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b24a4414-06f2-41ee-afd6-288669e544cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:05.178232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:05.185637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a4414-06f2-41ee-afd6-288669e544cc None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:05.191634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b810775-59e3-4f20-9d43-126dff8649a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:05.196252 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b810775-59e3-4f20-9d43-126dff8649a9 service swift] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:05.196454 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b810775-59e3-4f20-9d43-126dff8649a9 service swift] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:05.196702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b810775-59e3-4f20-9d43-126dff8649a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:05.206240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b810775-59e3-4f20-9d43-126dff8649a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:05.207860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b810775-59e3-4f20-9d43-126dff8649a9 service swift] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:05.218317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1115/2241] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:51:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 03:51:05.308347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824096843a544a0a8eb3120b662a9086'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:05.330488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824096843a544a0a8eb3120b662a9086'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:05.428049 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824096843a544a0a8eb3120b662a9086'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:05.429103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824096843a544a0a8eb3120b662a9086'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:05.441323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824096843a544a0a8eb3120b662a9086'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:05.447421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:05.459996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1127/2242] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:04 2022] DELETE /identity/v3/users/824096843a544a0a8eb3120b662a9086 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:05.464593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb8cff-a69b-44d1-91bf-e48bce668303 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:05.465161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb8cff-a69b-44d1-91bf-e48bce668303 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:05.465573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb8cff-a69b-44d1-91bf-e48bce668303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:05.520635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-0ebb8cff-a69b-44d1-91bf-e48bce668303 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:05.522871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0ebb8cff-a69b-44d1-91bf-e48bce668303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:05.705169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1116/2243] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:05.714121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:05.715942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:05.868863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b39a39-fe74-47a5-9874-2e137d30829f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:05.877061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77978fe4-c297-4348-8196-2592dd7d1835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:05.882749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77978fe4-c297-4348-8196-2592dd7d1835 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:05.882749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77978fe4-c297-4348-8196-2592dd7d1835 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:05.882749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77978fe4-c297-4348-8196-2592dd7d1835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:05.908092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77978fe4-c297-4348-8196-2592dd7d1835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:05.910102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77978fe4-c297-4348-8196-2592dd7d1835 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:05.945427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1128/2244] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:06.478209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb8cff-a69b-44d1-91bf-e48bce668303 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:06.480373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0ebb8cff-a69b-44d1-91bf-e48bce668303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:06.503718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb8cff-a69b-44d1-91bf-e48bce668303 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:06.510725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:06.511905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:06.512803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:06.513328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] PATH_INFO: `/v3/projects/0fd9cfe6621e4679a180dd73164123a8` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:06.515281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd9cfe6621e4679a180dd73164123a8)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:06.517686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:06.741067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd9cfe6621e4679a180dd73164123a8', 'request_id': 'req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:07.251027 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd9cfe6621e4679a180dd73164123a8', 'request_id': 'req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:07.290326 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd9cfe6621e4679a180dd73164123a8', 'request_id': 'req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:07.323488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] The token cache is being invalidate because project 0fd9cfe6621e4679a180dd73164123a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:07.326472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:07.329857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5b58e7a4-c1eb-4e31-b83a-28c8fcac0966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:07.336668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1117/2245] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:06 2022] DELETE /identity/v3/projects/0fd9cfe6621e4679a180dd73164123a8 => generated 0 bytes in 860 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:51:07.342266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6bacd-3599-4efa-8afc-559413179cb5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:07.342804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6bacd-3599-4efa-8afc-559413179cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:07.342894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6bacd-3599-4efa-8afc-559413179cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:07.412662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2ea6bacd-3599-4efa-8afc-559413179cb5 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:07.416432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2ea6bacd-3599-4efa-8afc-559413179cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:07.581335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1129/2246] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:07.590609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9229b-dfdb-496f-9c4b-14b61d74ef90 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:07.591725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9229b-dfdb-496f-9c4b-14b61d74ef90 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:07.592029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9229b-dfdb-496f-9c4b-14b61d74ef90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:07.617112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-c6b9229b-dfdb-496f-9c4b-14b61d74ef90 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:07.618366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c6b9229b-dfdb-496f-9c4b-14b61d74ef90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:07.632606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1118/2247] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:07.633662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6bacd-3599-4efa-8afc-559413179cb5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:07.646116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6bacd-3599-4efa-8afc-559413179cb5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:07.674945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ea45c-7190-47a6-82b2-e08afa6e6b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:07.674945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ea45c-7190-47a6-82b2-e08afa6e6b9f admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:07.674945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ea45c-7190-47a6-82b2-e08afa6e6b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:07.674945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ea45c-7190-47a6-82b2-e08afa6e6b9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:07.675450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ea45c-7190-47a6-82b2-e08afa6e6b9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:51:07.675450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ea45c-7190-47a6-82b2-e08afa6e6b9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:07.675450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ea45c-7190-47a6-82b2-e08afa6e6b9f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:07.698768 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1130/2248] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:51:07.700552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9229b-dfdb-496f-9c4b-14b61d74ef90 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:07.716120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9229b-dfdb-496f-9c4b-14b61d74ef90 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:07.728522 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23750752-33f1-47e1-ad27-5042571a2354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:07.732285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23750752-33f1-47e1-ad27-5042571a2354 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:07.732285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23750752-33f1-47e1-ad27-5042571a2354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:07.732285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23750752-33f1-47e1-ad27-5042571a2354 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:07.732285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23750752-33f1-47e1-ad27-5042571a2354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:07.733965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23750752-33f1-47e1-ad27-5042571a2354 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:07.926819 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1119/2249] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:07 2022] POST /identity/v3/projects => generated 377 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:07.932117 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ea45c-7190-47a6-82b2-e08afa6e6b9f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:07.947844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ea45c-7190-47a6-82b2-e08afa6e6b9f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:08.112037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bed03-34ba-4b61-bbc4-36b954e7e27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:08.113477 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bed03-34ba-4b61-bbc4-36b954e7e27d admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:08.116631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bed03-34ba-4b61-bbc4-36b954e7e27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:08.116631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bed03-34ba-4b61-bbc4-36b954e7e27d admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:08.116631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bed03-34ba-4b61-bbc4-36b954e7e27d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:08.121514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bed03-34ba-4b61-bbc4-36b954e7e27d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:08.266305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1131/2250] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:07 2022] POST /identity/v3/users => generated 289 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:51:08.267477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23750752-33f1-47e1-ad27-5042571a2354 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:08.290791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23750752-33f1-47e1-ad27-5042571a2354 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:08.314643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5bda1-061e-44fd-9bb8-5c6817536c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:08.315461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5bda1-061e-44fd-9bb8-5c6817536c05 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:08.315940 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5bda1-061e-44fd-9bb8-5c6817536c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:08.317417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5bda1-061e-44fd-9bb8-5c6817536c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:08.318584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5bda1-061e-44fd-9bb8-5c6817536c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:08.324326 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5bda1-061e-44fd-9bb8-5c6817536c05 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:08.403081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1120/2251] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:08.406879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bed03-34ba-4b61-bbc4-36b954e7e27d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:08.432155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bed03-34ba-4b61-bbc4-36b954e7e27d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:08.449784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bee17-71db-40cd-b1f9-086eae214e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:08.451112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bee17-71db-40cd-b1f9-086eae214e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:08.454045 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bee17-71db-40cd-b1f9-086eae214e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:08.454955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bee17-71db-40cd-b1f9-086eae214e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:08.456769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bee17-71db-40cd-b1f9-086eae214e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:08.460552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bee17-71db-40cd-b1f9-086eae214e2c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:08.478719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1132/2252] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:08.478993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5bda1-061e-44fd-9bb8-5c6817536c05 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:08.502574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5bda1-061e-44fd-9bb8-5c6817536c05 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:08.527175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a5c88-d4b3-4085-aef6-76462d227234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:08.528385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a5c88-d4b3-4085-aef6-76462d227234 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:08.528723 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a5c88-d4b3-4085-aef6-76462d227234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:08.529234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a5c88-d4b3-4085-aef6-76462d227234 admin admin] PATH_INFO: `/v3/projects/cb2d22c677bb48a1aa50631997fc0741/users/89faad7b1c26412cac5c2c624fa3f0ed/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:08.581180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a5c88-d4b3-4085-aef6-76462d227234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2d22c677bb48a1aa50631997fc0741, user_id=89faad7b1c26412cac5c2c624fa3f0ed, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:08.593812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a5c88-d4b3-4085-aef6-76462d227234 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:08.729313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1121/2253] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:08 2022] PUT /identity/v3/projects/cb2d22c677bb48a1aa50631997fc0741/users/89faad7b1c26412cac5c2c624fa3f0ed/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:08.735236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c47480-cf19-470a-8199-c0827e49a6a2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:08.735656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c47480-cf19-470a-8199-c0827e49a6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:08.736036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c47480-cf19-470a-8199-c0827e49a6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:08.817694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-11c47480-cf19-470a-8199-c0827e49a6a2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:08.824069 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-11c47480-cf19-470a-8199-c0827e49a6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:08.850061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:09.035722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1133/2254] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:09.058969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a5c88-d4b3-4085-aef6-76462d227234 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:09.062289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-520a5c88-d4b3-4085-aef6-76462d227234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:09.088241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:09.158763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a5c88-d4b3-4085-aef6-76462d227234 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:09.165905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1185cbf-a2be-44dc-9cbf-1f3c60282fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:09.166537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1185cbf-a2be-44dc-9cbf-1f3c60282fca service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:09.167183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1185cbf-a2be-44dc-9cbf-1f3c60282fca service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:09.167475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1185cbf-a2be-44dc-9cbf-1f3c60282fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:09.199428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1185cbf-a2be-44dc-9cbf-1f3c60282fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:09.200882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1185cbf-a2be-44dc-9cbf-1f3c60282fca service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:09.215305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1122/2255] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:12.271756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c47480-cf19-470a-8199-c0827e49a6a2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:12.273585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-11c47480-cf19-470a-8199-c0827e49a6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:12.302670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c47480-cf19-470a-8199-c0827e49a6a2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:12.312758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:12.313538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:12.313758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:12.314156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] PATH_INFO: `/v3/users/982649dd80c340c8b00b5d795504f9b8` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:12.323762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=982649dd80c340c8b00b5d795504f9b8)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:12.325851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:12.619642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982649dd80c340c8b00b5d795504f9b8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:12.643587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982649dd80c340c8b00b5d795504f9b8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:12.738619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982649dd80c340c8b00b5d795504f9b8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:12.740051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982649dd80c340c8b00b5d795504f9b8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:12.759155 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982649dd80c340c8b00b5d795504f9b8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:12.771422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1134/2256] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:12 2022] DELETE /identity/v3/users/982649dd80c340c8b00b5d795504f9b8 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:12.778097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e4e61-6090-4698-8e70-fd8243ffd52e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:12.778964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e4e61-6090-4698-8e70-fd8243ffd52e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:12.779391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e4e61-6090-4698-8e70-fd8243ffd52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:12.862253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-0e0e4e61-6090-4698-8e70-fd8243ffd52e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:12.867898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0e0e4e61-6090-4698-8e70-fd8243ffd52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:13.116511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1123/2257] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:13.133945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:13.135404 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:13.287742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58775294-d151-4d66-a43b-a16765141e6e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:13.302744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac4dc5-18de-4a22-8b2f-d5711cbd3279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:13.303390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac4dc5-18de-4a22-8b2f-d5711cbd3279 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:13.303667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac4dc5-18de-4a22-8b2f-d5711cbd3279 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:13.303941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac4dc5-18de-4a22-8b2f-d5711cbd3279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:13.338606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac4dc5-18de-4a22-8b2f-d5711cbd3279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:13.340426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac4dc5-18de-4a22-8b2f-d5711cbd3279 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:13.365280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1135/2258] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:15.319532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e4e61-6090-4698-8e70-fd8243ffd52e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:15.320944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0e0e4e61-6090-4698-8e70-fd8243ffd52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:15.337688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e4e61-6090-4698-8e70-fd8243ffd52e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:15.394947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:15.394947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:15.394947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:15.394947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] PATH_INFO: `/v3/projects/0e3254e44c8f435b9fbfc33bf5e12381` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:15.395301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e3254e44c8f435b9fbfc33bf5e12381)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:15.395301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:15.833460 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e3254e44c8f435b9fbfc33bf5e12381', 'request_id': 'req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:15.948834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e3254e44c8f435b9fbfc33bf5e12381', 'request_id': 'req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:15.954362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e3254e44c8f435b9fbfc33bf5e12381', 'request_id': 'req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:15.986395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] The token cache is being invalidate because project 0e3254e44c8f435b9fbfc33bf5e12381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:15.987101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:15.987778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:15.990417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1124/2259] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:15 2022] DELETE /identity/v3/projects/0e3254e44c8f435b9fbfc33bf5e12381 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:15.994211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac4dc5-18de-4a22-8b2f-d5711cbd3279 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:16.033625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac4dc5-18de-4a22-8b2f-d5711cbd3279 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:16.083433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:16.199411 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:16.200176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:16.200682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:16.201156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] PATH_INFO: `/v3/users/e4fbe98b7df04376a13ac8c6ce9fa4ce` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:16.203175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4fbe98b7df04376a13ac8c6ce9fa4ce)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:16.209324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:16.556890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4fbe98b7df04376a13ac8c6ce9fa4ce'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:16.585082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4fbe98b7df04376a13ac8c6ce9fa4ce'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:16.687866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4fbe98b7df04376a13ac8c6ce9fa4ce'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:16.689044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4fbe98b7df04376a13ac8c6ce9fa4ce'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:16.705919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6bcca9c3-ba41-4e50-b9c0-033588a4cbed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4fbe98b7df04376a13ac8c6ce9fa4ce'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:16.729852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1136/2260] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:15 2022] DELETE /identity/v3/users/e4fbe98b7df04376a13ac8c6ce9fa4ce => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:18.195257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:18.197491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:18.226221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6a99d-31e6-4cfa-b776-42cecf7b6679 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:18.299747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:18.372875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:18.372875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:18.372875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:18.372875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] PATH_INFO: `/v3/projects/5ab0852151984ddb96cb1e0d8f510a4c` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:18.373562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab0852151984ddb96cb1e0d8f510a4c)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:18.374177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:18.593968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab0852151984ddb96cb1e0d8f510a4c', 'request_id': 'req-37274895-7e53-4e52-8d3f-6d5e933a0a91'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:18.711120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab0852151984ddb96cb1e0d8f510a4c', 'request_id': 'req-37274895-7e53-4e52-8d3f-6d5e933a0a91'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:18.719849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab0852151984ddb96cb1e0d8f510a4c', 'request_id': 'req-37274895-7e53-4e52-8d3f-6d5e933a0a91'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:18.745260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] The token cache is being invalidate because project 5ab0852151984ddb96cb1e0d8f510a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:18.745561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:18.746481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37274895-7e53-4e52-8d3f-6d5e933a0a91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:18.749921 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1125/2261] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:18 2022] DELETE /identity/v3/projects/5ab0852151984ddb96cb1e0d8f510a4c => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:18.757253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15581269-ab57-44f0-a858-ba7eb78a7952 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:18.759076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15581269-ab57-44f0-a858-ba7eb78a7952 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:18.759778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15581269-ab57-44f0-a858-ba7eb78a7952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:18.824952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-15581269-ab57-44f0-a858-ba7eb78a7952 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:18.826907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-15581269-ab57-44f0-a858-ba7eb78a7952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:19.093407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9da78-923b-4b1f-a078-4c61277c6998 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:19.093735 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9da78-923b-4b1f-a078-4c61277c6998 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:19.093972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9da78-923b-4b1f-a078-4c61277c6998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:19.098011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1137/2262] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:19.120148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-95e9da78-923b-4b1f-a078-4c61277c6998 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:19.120527 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-95e9da78-923b-4b1f-a078-4c61277c6998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:19.135629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15581269-ab57-44f0-a858-ba7eb78a7952 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:19.148121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1126/2263] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:19.149549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15581269-ab57-44f0-a858-ba7eb78a7952 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:19.159261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e095a-cc4d-4be5-be23-365af4d98712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:19.159599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e095a-cc4d-4be5-be23-365af4d98712 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:19.160063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e095a-cc4d-4be5-be23-365af4d98712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:19.160205 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e095a-cc4d-4be5-be23-365af4d98712 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:19.162331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e095a-cc4d-4be5-be23-365af4d98712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:51:19.162939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e095a-cc4d-4be5-be23-365af4d98712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:19.166676 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e095a-cc4d-4be5-be23-365af4d98712 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:19.218423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1138/2264] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:51:19.221322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9da78-923b-4b1f-a078-4c61277c6998 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:19.243492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9da78-923b-4b1f-a078-4c61277c6998 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:19.254283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd034b9-fca9-420d-aa6b-7cff31bfd335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:19.255741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd034b9-fca9-420d-aa6b-7cff31bfd335 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:19.256172 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd034b9-fca9-420d-aa6b-7cff31bfd335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:19.256842 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd034b9-fca9-420d-aa6b-7cff31bfd335 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:19.257705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd034b9-fca9-420d-aa6b-7cff31bfd335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:19.260670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd034b9-fca9-420d-aa6b-7cff31bfd335 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:19.430251 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1127/2265] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:19 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:19.431958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e095a-cc4d-4be5-be23-365af4d98712 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:19.446099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e095a-cc4d-4be5-be23-365af4d98712 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:19.533179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:19.598625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a63580-cf96-4b46-9d16-9b68efadf37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:19.599668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a63580-cf96-4b46-9d16-9b68efadf37a admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:19.600113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a63580-cf96-4b46-9d16-9b68efadf37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:19.600570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a63580-cf96-4b46-9d16-9b68efadf37a admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:19.601365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a63580-cf96-4b46-9d16-9b68efadf37a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:19.605933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a63580-cf96-4b46-9d16-9b68efadf37a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:19.730523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1139/2266] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:19 2022] POST /identity/v3/users => generated 291 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:19.736099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd034b9-fca9-420d-aa6b-7cff31bfd335 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:19.761973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd034b9-fca9-420d-aa6b-7cff31bfd335 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:19.791715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e0ad6-4f70-4bf4-bb44-d118a0b9e4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:19.801214 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e0ad6-4f70-4bf4-bb44-d118a0b9e4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:19.801567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e0ad6-4f70-4bf4-bb44-d118a0b9e4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:19.806270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e0ad6-4f70-4bf4-bb44-d118a0b9e4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:19.806968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e0ad6-4f70-4bf4-bb44-d118a0b9e4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:19.812746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e0ad6-4f70-4bf4-bb44-d118a0b9e4d7 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:19.845420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1128/2267] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:19.850182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a63580-cf96-4b46-9d16-9b68efadf37a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:19.862224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a63580-cf96-4b46-9d16-9b68efadf37a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:19.898274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c3676-ba1d-454c-aed8-8544ec919d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:19.901888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c3676-ba1d-454c-aed8-8544ec919d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:19.904040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c3676-ba1d-454c-aed8-8544ec919d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:19.904040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c3676-ba1d-454c-aed8-8544ec919d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:19.905503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c3676-ba1d-454c-aed8-8544ec919d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:19.908458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c3676-ba1d-454c-aed8-8544ec919d4b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:19.936973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1140/2268] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:19.940893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e0ad6-4f70-4bf4-bb44-d118a0b9e4d7 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:19.968305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e0ad6-4f70-4bf4-bb44-d118a0b9e4d7 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:19.988344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac24019-69c0-4235-abdf-97bb3d3e0a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:19.989021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac24019-69c0-4235-abdf-97bb3d3e0a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:19.989304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac24019-69c0-4235-abdf-97bb3d3e0a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:19.989656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac24019-69c0-4235-abdf-97bb3d3e0a35 admin admin] PATH_INFO: `/v3/projects/d8a24babdc084ea396966d36680fbcec/users/0baf3363a648444d941489b176b97166/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:20.033194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899c5c1-7d1a-433d-ad96-890e5a800782 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:20.034078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899c5c1-7d1a-433d-ad96-890e5a800782 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:20.034190 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899c5c1-7d1a-433d-ad96-890e5a800782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:20.055981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac24019-69c0-4235-abdf-97bb3d3e0a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a24babdc084ea396966d36680fbcec, user_id=0baf3363a648444d941489b176b97166, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:20.059687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac24019-69c0-4235-abdf-97bb3d3e0a35 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:20.089165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e899c5c1-7d1a-433d-ad96-890e5a800782 None None] MFA Rules not processed for user `89faad7b1c26412cac5c2c624fa3f0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:20.091334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e899c5c1-7d1a-433d-ad96-890e5a800782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:20.203225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1129/2269] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:19 2022] PUT /identity/v3/projects/d8a24babdc084ea396966d36680fbcec/users/0baf3363a648444d941489b176b97166/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:20.217393 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21dd42-f52a-4aaf-bd3b-cd6cd419706a None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:20.217682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21dd42-f52a-4aaf-bd3b-cd6cd419706a None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:20.218060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21dd42-f52a-4aaf-bd3b-cd6cd419706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:20.277655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1141/2270] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:20.278851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899c5c1-7d1a-433d-ad96-890e5a800782 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:20.280253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e899c5c1-7d1a-433d-ad96-890e5a800782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:20.296975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899c5c1-7d1a-433d-ad96-890e5a800782 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:20.343638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-bb21dd42-f52a-4aaf-bd3b-cd6cd419706a None None] MFA Rules not processed for user `0baf3363a648444d941489b176b97166`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:20.345230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bb21dd42-f52a-4aaf-bd3b-cd6cd419706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:20.383743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe09da-f385-49cc-9c65-bae2b89d5573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:20.384503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe09da-f385-49cc-9c65-bae2b89d5573 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:20.384761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe09da-f385-49cc-9c65-bae2b89d5573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:20.385032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe09da-f385-49cc-9c65-bae2b89d5573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:20.385690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe09da-f385-49cc-9c65-bae2b89d5573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:20.399652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe09da-f385-49cc-9c65-bae2b89d5573 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:20.502543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:20.554246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1130/2271] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:20.562410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21dd42-f52a-4aaf-bd3b-cd6cd419706a None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:20.570958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21dd42-f52a-4aaf-bd3b-cd6cd419706a None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:20.579554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e5315-339b-4e94-9bf3-273025aeae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:20.581197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e5315-339b-4e94-9bf3-273025aeae06 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:20.581540 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e5315-339b-4e94-9bf3-273025aeae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:20.581939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e5315-339b-4e94-9bf3-273025aeae06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:20.582653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e5315-339b-4e94-9bf3-273025aeae06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:20.585894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e5315-339b-4e94-9bf3-273025aeae06 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:20.626314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1142/2272] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:20 2022] POST /identity/v3/projects => generated 377 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:20.627910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe09da-f385-49cc-9c65-bae2b89d5573 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:20.742413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1131/2273] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:20 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:20.742413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e5315-339b-4e94-9bf3-273025aeae06 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:20.754946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:20.819634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe09da-f385-49cc-9c65-bae2b89d5573 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:20.885924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e5315-339b-4e94-9bf3-273025aeae06 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:20.898190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f47896-c660-4797-9f5f-0279817eeb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:20.898873 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f47896-c660-4797-9f5f-0279817eeb94 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:20.899096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f47896-c660-4797-9f5f-0279817eeb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:20.899393 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f47896-c660-4797-9f5f-0279817eeb94 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:20.900124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f47896-c660-4797-9f5f-0279817eeb94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:20.902954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f47896-c660-4797-9f5f-0279817eeb94 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:20.940972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a335fe-4144-4181-82f6-125a26f3a36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:20.941710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a335fe-4144-4181-82f6-125a26f3a36a admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:20.942029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a335fe-4144-4181-82f6-125a26f3a36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:20.942308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a335fe-4144-4181-82f6-125a26f3a36a admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:20.943704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a335fe-4144-4181-82f6-125a26f3a36a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:20.946559 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a335fe-4144-4181-82f6-125a26f3a36a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:21.040192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1143/2274] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:20 2022] POST /identity/v3/users => generated 295 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:21.044299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a335fe-4144-4181-82f6-125a26f3a36a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:21.056114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1132/2275] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:20 2022] POST /identity/v3/users => generated 297 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:21.057936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a335fe-4144-4181-82f6-125a26f3a36a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:21.059998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f47896-c660-4797-9f5f-0279817eeb94 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:21.070415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b17d9-9868-4867-bf66-6fc5ece8339b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:21.071272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b17d9-9868-4867-bf66-6fc5ece8339b admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:21.074969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b17d9-9868-4867-bf66-6fc5ece8339b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:21.075381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b17d9-9868-4867-bf66-6fc5ece8339b admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:21.076408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b17d9-9868-4867-bf66-6fc5ece8339b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:21.078571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b17d9-9868-4867-bf66-6fc5ece8339b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:21.088536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f47896-c660-4797-9f5f-0279817eeb94 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:21.096798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1144/2276] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:21.098209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b17d9-9868-4867-bf66-6fc5ece8339b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:21.105040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a20f8-2288-47d2-bd8c-1b85b9fce3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:21.106059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a20f8-2288-47d2-bd8c-1b85b9fce3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:21.107210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a20f8-2288-47d2-bd8c-1b85b9fce3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:21.107505 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a20f8-2288-47d2-bd8c-1b85b9fce3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:21.109463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a20f8-2288-47d2-bd8c-1b85b9fce3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:21.111273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a20f8-2288-47d2-bd8c-1b85b9fce3e6 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:21.112171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b17d9-9868-4867-bf66-6fc5ece8339b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:21.118665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1130010-1df9-4b73-9fe1-2d504cc7bb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:21.120291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1130010-1df9-4b73-9fe1-2d504cc7bb7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:21.123694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1130010-1df9-4b73-9fe1-2d504cc7bb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:21.123694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1130010-1df9-4b73-9fe1-2d504cc7bb7d admin admin] PATH_INFO: `/v3/projects/ac0ebe0ca1bd4032b42483825e838762/users/5a9252ac49fc4c79b7bd5e6fc425e1a2/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:21.128425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1133/2277] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:21.134770 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a20f8-2288-47d2-bd8c-1b85b9fce3e6 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:21.217369 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1130010-1df9-4b73-9fe1-2d504cc7bb7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0ebe0ca1bd4032b42483825e838762, user_id=5a9252ac49fc4c79b7bd5e6fc425e1a2, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:21.234712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1130010-1df9-4b73-9fe1-2d504cc7bb7d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:21.274609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a20f8-2288-47d2-bd8c-1b85b9fce3e6 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:21.283062 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6376-4179-4e58-8aea-0d7dd5868410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:21.284102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6376-4179-4e58-8aea-0d7dd5868410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:21.284556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6376-4179-4e58-8aea-0d7dd5868410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:21.285049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6376-4179-4e58-8aea-0d7dd5868410 admin admin] PATH_INFO: `/v3/projects/cea423049d40432abc2b3dc512dc3016/users/f8d41f19494b47d0935c496f603d2960/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:21.352658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f6376-4179-4e58-8aea-0d7dd5868410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea423049d40432abc2b3dc512dc3016, user_id=f8d41f19494b47d0935c496f603d2960, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:21.360257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f6376-4179-4e58-8aea-0d7dd5868410 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:21.602878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1134/2278] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:21 2022] PUT /identity/v3/projects/cea423049d40432abc2b3dc512dc3016/users/f8d41f19494b47d0935c496f603d2960/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:21.607187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6376-4179-4e58-8aea-0d7dd5868410 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:21.620546 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1145/2279] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:21 2022] PUT /identity/v3/projects/ac0ebe0ca1bd4032b42483825e838762/users/5a9252ac49fc4c79b7bd5e6fc425e1a2/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:21.625687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1130010-1df9-4b73-9fe1-2d504cc7bb7d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:21.846004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1130010-1df9-4b73-9fe1-2d504cc7bb7d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:21.859782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6376-4179-4e58-8aea-0d7dd5868410 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:21.861674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3165d3-e522-45c8-b29a-39a8ceb042a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:21.862808 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3165d3-e522-45c8-b29a-39a8ceb042a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:21.864078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3165d3-e522-45c8-b29a-39a8ceb042a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:21.864857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3165d3-e522-45c8-b29a-39a8ceb042a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:21.869853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3165d3-e522-45c8-b29a-39a8ceb042a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:21.871944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3165d3-e522-45c8-b29a-39a8ceb042a6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:21.891520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d167bd6-231b-40e3-b67f-2d5fb9b8270b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:21.894597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167bd6-231b-40e3-b67f-2d5fb9b8270b admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:21.895080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167bd6-231b-40e3-b67f-2d5fb9b8270b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:21.896035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167bd6-231b-40e3-b67f-2d5fb9b8270b admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:21.896229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1146/2280] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:21.903365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d167bd6-231b-40e3-b67f-2d5fb9b8270b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:21.903701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3165d3-e522-45c8-b29a-39a8ceb042a6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:21.907623 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d167bd6-231b-40e3-b67f-2d5fb9b8270b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:21.934462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3165d3-e522-45c8-b29a-39a8ceb042a6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:21.940131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1135/2281] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:21.947195 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d167bd6-231b-40e3-b67f-2d5fb9b8270b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:21.954557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78f0d9-e434-45ea-a998-d3b3b73b51d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:21.960811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78f0d9-e434-45ea-a998-d3b3b73b51d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:21.960811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78f0d9-e434-45ea-a998-d3b3b73b51d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:21.960811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78f0d9-e434-45ea-a998-d3b3b73b51d6 admin admin] PATH_INFO: `/v3/projects/ac0ebe0ca1bd4032b42483825e838762/users/5a9252ac49fc4c79b7bd5e6fc425e1a2/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:21.960811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78f0d9-e434-45ea-a998-d3b3b73b51d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0ebe0ca1bd4032b42483825e838762, user_id=5a9252ac49fc4c79b7bd5e6fc425e1a2, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:21.964512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78f0d9-e434-45ea-a998-d3b3b73b51d6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:21.965586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d167bd6-231b-40e3-b67f-2d5fb9b8270b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:21.972987 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeaa423-af6a-4b5f-957e-2e846249fb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:21.974618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeaa423-af6a-4b5f-957e-2e846249fb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:21.974927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeaa423-af6a-4b5f-957e-2e846249fb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:21.975191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeaa423-af6a-4b5f-957e-2e846249fb59 admin admin] PATH_INFO: `/v3/projects/cea423049d40432abc2b3dc512dc3016/users/f8d41f19494b47d0935c496f603d2960/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:21.978323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeaa423-af6a-4b5f-957e-2e846249fb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea423049d40432abc2b3dc512dc3016, user_id=f8d41f19494b47d0935c496f603d2960, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:21.982683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeaa423-af6a-4b5f-957e-2e846249fb59 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:21.984001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1147/2282] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:21 2022] PUT /identity/v3/projects/ac0ebe0ca1bd4032b42483825e838762/users/5a9252ac49fc4c79b7bd5e6fc425e1a2/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:21.998447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1136/2283] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:21 2022] PUT /identity/v3/projects/cea423049d40432abc2b3dc512dc3016/users/f8d41f19494b47d0935c496f603d2960/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:22.008202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d3659-119e-4117-b491-94a7395a6e1e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:22.008498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d3659-119e-4117-b491-94a7395a6e1e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:22.008783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d3659-119e-4117-b491-94a7395a6e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:22.052763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:22.092599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f67d3659-119e-4117-b491-94a7395a6e1e None None] MFA Rules not processed for user `f8d41f19494b47d0935c496f603d2960`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:22.094651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f67d3659-119e-4117-b491-94a7395a6e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:22.278924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1148/2284] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:22.287253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeaa423-af6a-4b5f-957e-2e846249fb59 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:22.292504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-fdeaa423-af6a-4b5f-957e-2e846249fb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:22.314865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:22.457277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeaa423-af6a-4b5f-957e-2e846249fb59 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:22.473030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af29d52-5df7-4c60-b80a-96c2607d0698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:22.478007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af29d52-5df7-4c60-b80a-96c2607d0698 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:22.480691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af29d52-5df7-4c60-b80a-96c2607d0698 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:22.482558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af29d52-5df7-4c60-b80a-96c2607d0698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:22.541182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af29d52-5df7-4c60-b80a-96c2607d0698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:22.547003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af29d52-5df7-4c60-b80a-96c2607d0698 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:22.590210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1137/2285] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:22.863111 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d3659-119e-4117-b491-94a7395a6e1e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:22.882225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d3659-119e-4117-b491-94a7395a6e1e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:22.900377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3e862-2865-46aa-b172-fa76322e1199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:22.904709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3e862-2865-46aa-b172-fa76322e1199 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:22.904709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3e862-2865-46aa-b172-fa76322e1199 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:22.905080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3e862-2865-46aa-b172-fa76322e1199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:23.062005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3e862-2865-46aa-b172-fa76322e1199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:23.064295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3e862-2865-46aa-b172-fa76322e1199 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:23.083997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1149/2286] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:51:23.848127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5330-9efd-4747-a932-a438ba104803 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:23.849625 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5330-9efd-4747-a932-a438ba104803 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:23.849625 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5330-9efd-4747-a932-a438ba104803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:23.894045 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4a5e5330-9efd-4747-a932-a438ba104803 None None] MFA Rules not processed for user `26995363963a4a52acc3f57ce1575e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:23.895332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4a5e5330-9efd-4747-a932-a438ba104803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:24.044759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1138/2287] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:25.934870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbda313-1856-4237-a008-ad567c617335 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:25.934870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbda313-1856-4237-a008-ad567c617335 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:25.934870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbda313-1856-4237-a008-ad567c617335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:25.985095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-bbbda313-1856-4237-a008-ad567c617335 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:25.986729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bbbda313-1856-4237-a008-ad567c617335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:26.152992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e5330-9efd-4747-a932-a438ba104803 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:26.154608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4a5e5330-9efd-4747-a932-a438ba104803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:26.157505 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1150/2288] 10.222.0.42 () {58 vars in 967 bytes} [Sat Sep 17 03:51:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:26.371325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e5330-9efd-4747-a932-a438ba104803 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:26.380013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fedf7c-545e-4bda-8181-6f0e1a6694bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:26.380691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fedf7c-545e-4bda-8181-6f0e1a6694bd service swift] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:26.380979 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fedf7c-545e-4bda-8181-6f0e1a6694bd service swift] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:26.381258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fedf7c-545e-4bda-8181-6f0e1a6694bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:26.390939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fedf7c-545e-4bda-8181-6f0e1a6694bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:26.393199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fedf7c-545e-4bda-8181-6f0e1a6694bd service swift] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:26.403703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1139/2289] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:51:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:51:30.670164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5955-9875-48ff-9def-68c1b717e698 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:30.670629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5955-9875-48ff-9def-68c1b717e698 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:30.670722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5955-9875-48ff-9def-68c1b717e698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:30.714739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c06c5955-9875-48ff-9def-68c1b717e698 None None] MFA Rules not processed for user `5a9252ac49fc4c79b7bd5e6fc425e1a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:30.716647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c06c5955-9875-48ff-9def-68c1b717e698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:30.747025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:30.817491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1151/2290] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:30.834658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fedf7c-545e-4bda-8181-6f0e1a6694bd service swift] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:30.834658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-41fedf7c-545e-4bda-8181-6f0e1a6694bd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:30.896395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:30.930129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fedf7c-545e-4bda-8181-6f0e1a6694bd service swift] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:30.937440 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c72c23-1fcd-4ca2-8309-4915bd61f8dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:30.938686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c72c23-1fcd-4ca2-8309-4915bd61f8dd service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:30.939139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c72c23-1fcd-4ca2-8309-4915bd61f8dd service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:30.939756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c72c23-1fcd-4ca2-8309-4915bd61f8dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:31.014503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c72c23-1fcd-4ca2-8309-4915bd61f8dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:31.016293 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c72c23-1fcd-4ca2-8309-4915bd61f8dd service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:31.026669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1140/2291] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:31.681808 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c5955-9875-48ff-9def-68c1b717e698 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:31.773592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c5955-9875-48ff-9def-68c1b717e698 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:31.789148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:31.789853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:31.790142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:31.793438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] PATH_INFO: `/v3/users/0baf3363a648444d941489b176b97166` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:31.794868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0baf3363a648444d941489b176b97166)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:31.796877 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:32.046165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0baf3363a648444d941489b176b97166'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:32.061987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0baf3363a648444d941489b176b97166'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:32.170297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0baf3363a648444d941489b176b97166'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:32.170945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0baf3363a648444d941489b176b97166'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:32.180210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0baf3363a648444d941489b176b97166'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:32.187152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1152/2292] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:31 2022] DELETE /identity/v3/users/0baf3363a648444d941489b176b97166 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:32.191850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1391d-ebc3-4d18-9675-248c53d030d1 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:32.192389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1391d-ebc3-4d18-9675-248c53d030d1 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:32.192825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1391d-ebc3-4d18-9675-248c53d030d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:32.242133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-cad1391d-ebc3-4d18-9675-248c53d030d1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:32.243513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-cad1391d-ebc3-4d18-9675-248c53d030d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:32.331287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1141/2293] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:32.344923 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:32.346528 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:32.355109 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e505258-0fb9-43fe-bc19-a7301fe6829e None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:32.355559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e505258-0fb9-43fe-bc19-a7301fe6829e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:32.355978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e505258-0fb9-43fe-bc19-a7301fe6829e None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:32.357327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1142/2294] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:51:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 03:51:32.365909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1014959-0e21-4ba3-9952-a4ab6df032fe None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:32.366436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1014959-0e21-4ba3-9952-a4ab6df032fe None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:32.366738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1014959-0e21-4ba3-9952-a4ab6df032fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:32.395982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b1014959-0e21-4ba3-9952-a4ab6df032fe None None] MFA Rules not processed for user `34ec5c9c71d14762a4869b7d66e87ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:32.398484 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b1014959-0e21-4ba3-9952-a4ab6df032fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:32.426318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:32.439139 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a650d94-7f1b-44eb-a278-0c9107a2eca4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:32.444028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de8eb0-51f8-4c30-a436-b6a91e4b88ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:32.444634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de8eb0-51f8-4c30-a436-b6a91e4b88ac service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:32.444910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de8eb0-51f8-4c30-a436-b6a91e4b88ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:32.445183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de8eb0-51f8-4c30-a436-b6a91e4b88ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:32.459052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de8eb0-51f8-4c30-a436-b6a91e4b88ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:32.463866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de8eb0-51f8-4c30-a436-b6a91e4b88ac service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:32.497424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1143/2295] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:51:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:32.497925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1153/2296] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:32.509909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f7c2d-8080-44e4-a494-1ff542418651 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:32.511047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f7c2d-8080-44e4-a494-1ff542418651 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:32.511340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f7c2d-8080-44e4-a494-1ff542418651 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:32.518322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1144/2297] 10.222.0.42 () {54 vars in 961 bytes} [Sat Sep 17 03:51:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:51:32.526018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb16839-762a-4293-8a89-c789dd24f764 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:32.527200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb16839-762a-4293-8a89-c789dd24f764 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:32.527731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb16839-762a-4293-8a89-c789dd24f764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:32.557484 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f7c2d-8080-44e4-a494-1ff542418651 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:32.560337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bf2f7c2d-8080-44e4-a494-1ff542418651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:32.597671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8fb16839-762a-4293-8a89-c789dd24f764 None None] MFA Rules not processed for user `34ec5c9c71d14762a4869b7d66e87ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:32.613865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f7c2d-8080-44e4-a494-1ff542418651 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:32.614367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8fb16839-762a-4293-8a89-c789dd24f764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:32.614683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:32.614683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:32.614683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:32.614683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] PATH_INFO: `/v3/users/0713ce790d0f46678e6402e290d511b5` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:32.615147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0713ce790d0f46678e6402e290d511b5)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:32.615226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1154/2298] 10.222.0.42 () {58 vars in 1058 bytes} [Sat Sep 17 03:51:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:32.615300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:32.625813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:32.634170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb16839-762a-4293-8a89-c789dd24f764 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:32.658161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb16839-762a-4293-8a89-c789dd24f764 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:32.677518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01c350-6ffa-49e8-a6ce-f6f96779cd7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ec5c9c71d14762a4869b7d66e87ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:32.678384 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01c350-6ffa-49e8-a6ce-f6f96779cd7d service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:32.678801 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01c350-6ffa-49e8-a6ce-f6f96779cd7d service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:32.679006 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01c350-6ffa-49e8-a6ce-f6f96779cd7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:32.703894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01c350-6ffa-49e8-a6ce-f6f96779cd7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:32.705747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01c350-6ffa-49e8-a6ce-f6f96779cd7d service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:32.723411 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1155/2299] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:32.854040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0713ce790d0f46678e6402e290d511b5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:32.947923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0713ce790d0f46678e6402e290d511b5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:32.948825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0713ce790d0f46678e6402e290d511b5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:32.956407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0713ce790d0f46678e6402e290d511b5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:32.980619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0713ce790d0f46678e6402e290d511b5'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:32.989014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1145/2300] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:32 2022] DELETE /identity/v3/users/0713ce790d0f46678e6402e290d511b5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:33.453535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01c350-6ffa-49e8-a6ce-f6f96779cd7d service placement] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:33.467295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01c350-6ffa-49e8-a6ce-f6f96779cd7d service placement] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:33.589040 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:33.591586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:33.591586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:33.591586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] PATH_INFO: `/v3/projects/ca2e65ee55f142dd9ad4419ab18dcf7d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:33.592303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca2e65ee55f142dd9ad4419ab18dcf7d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:33.594497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:33.750938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:33.752550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:33.773717 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624bb16-1434-4555-b128-3c2b86cd0c98 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:33.776153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca2e65ee55f142dd9ad4419ab18dcf7d', 'request_id': 'req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:33.791696 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca2e65ee55f142dd9ad4419ab18dcf7d', 'request_id': 'req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:33.857597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:33.864794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:33.864794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:33.864794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] PATH_INFO: `/v3/projects/d8a24babdc084ea396966d36680fbcec` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:33.864794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a24babdc084ea396966d36680fbcec)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:33.864794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:33.919234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca2e65ee55f142dd9ad4419ab18dcf7d', 'request_id': 'req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:33.934472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] The token cache is being invalidate because project ca2e65ee55f142dd9ad4419ab18dcf7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:33.935068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:33.936036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8315ee6d-46ee-4bd1-aba7-200489f4b6ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:33.938462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1156/2301] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:33 2022] DELETE /identity/v3/projects/ca2e65ee55f142dd9ad4419ab18dcf7d => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:33.944048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a852144-917e-46be-8e6d-e0dcf7bad3bb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:33.944333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a852144-917e-46be-8e6d-e0dcf7bad3bb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:33.944631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a852144-917e-46be-8e6d-e0dcf7bad3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:33.966175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8a852144-917e-46be-8e6d-e0dcf7bad3bb None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:33.967495 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8a852144-917e-46be-8e6d-e0dcf7bad3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:33.985082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1157/2302] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:33.990545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b075ae7-b75f-4d13-8cf0-29a179e9c612 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:33.990545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b075ae7-b75f-4d13-8cf0-29a179e9c612 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:33.990545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b075ae7-b75f-4d13-8cf0-29a179e9c612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:34.011969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0b075ae7-b75f-4d13-8cf0-29a179e9c612 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:34.013241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0b075ae7-b75f-4d13-8cf0-29a179e9c612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:34.027360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1158/2303] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:34.032547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b075ae7-b75f-4d13-8cf0-29a179e9c612 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:34.065545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b075ae7-b75f-4d13-8cf0-29a179e9c612 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:34.077671 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a24babdc084ea396966d36680fbcec', 'request_id': 'req-0184a515-15e0-4e9d-9378-6da22a9e47a8'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:34.128316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:34.152210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7a137-a8aa-4b19-9033-b883ecf6c0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:34.153210 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7a137-a8aa-4b19-9033-b883ecf6c0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:34.153678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7a137-a8aa-4b19-9033-b883ecf6c0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:34.154207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7a137-a8aa-4b19-9033-b883ecf6c0a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:34.155021 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7a137-a8aa-4b19-9033-b883ecf6c0a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:51:34.155661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7a137-a8aa-4b19-9033-b883ecf6c0a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:34.159044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7a137-a8aa-4b19-9033-b883ecf6c0a6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:34.193223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1159/2304] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:51:34.198486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7a137-a8aa-4b19-9033-b883ecf6c0a6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:34.214334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7a137-a8aa-4b19-9033-b883ecf6c0a6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:34.220400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337dd22-e372-4403-8299-893815dd43c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:34.221232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337dd22-e372-4403-8299-893815dd43c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:34.221939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337dd22-e372-4403-8299-893815dd43c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:34.222374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337dd22-e372-4403-8299-893815dd43c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:34.223165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337dd22-e372-4403-8299-893815dd43c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:34.225884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337dd22-e372-4403-8299-893815dd43c1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:34.276256 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a24babdc084ea396966d36680fbcec', 'request_id': 'req-0184a515-15e0-4e9d-9378-6da22a9e47a8'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:34.286525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a24babdc084ea396966d36680fbcec', 'request_id': 'req-0184a515-15e0-4e9d-9378-6da22a9e47a8'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:34.309071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] The token cache is being invalidate because project d8a24babdc084ea396966d36680fbcec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:34.309369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:34.310127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:34.311996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1146/2305] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:33 2022] DELETE /identity/v3/projects/d8a24babdc084ea396966d36680fbcec => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:51:34.315683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:34.319084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:34.334246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184a515-15e0-4e9d-9378-6da22a9e47a8 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:34.340845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:34.341853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:34.348870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:34.348870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] PATH_INFO: `/v3/users/f8d41f19494b47d0935c496f603d2960` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:34.348870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d41f19494b47d0935c496f603d2960)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:34.348870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:34.386335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1160/2306] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:34 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:34.391977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337dd22-e372-4403-8299-893815dd43c1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:34.393857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c337dd22-e372-4403-8299-893815dd43c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:34.410214 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:34.559645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337dd22-e372-4403-8299-893815dd43c1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:34.567167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83081f-1952-442f-8d0c-1db30d5addc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:34.568500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83081f-1952-442f-8d0c-1db30d5addc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:34.569515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83081f-1952-442f-8d0c-1db30d5addc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:34.569916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83081f-1952-442f-8d0c-1db30d5addc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:34.579366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83081f-1952-442f-8d0c-1db30d5addc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:34.583942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83081f-1952-442f-8d0c-1db30d5addc7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:34.689469 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1161/2307] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:34 2022] POST /identity/v3/users => generated 286 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:34.693589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83081f-1952-442f-8d0c-1db30d5addc7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:34.695728 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d41f19494b47d0935c496f603d2960'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:34.720389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83081f-1952-442f-8d0c-1db30d5addc7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:34.730598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c532537-b8bc-41a3-9c01-084728939807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:34.731458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532537-b8bc-41a3-9c01-084728939807 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:34.732126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532537-b8bc-41a3-9c01-084728939807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:34.732654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532537-b8bc-41a3-9c01-084728939807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:34.733867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c532537-b8bc-41a3-9c01-084728939807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:34.738340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c532537-b8bc-41a3-9c01-084728939807 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:34.749852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1162/2308] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:34.757938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c532537-b8bc-41a3-9c01-084728939807 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:34.773477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d41f19494b47d0935c496f603d2960'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:34.774173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d41f19494b47d0935c496f603d2960'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:34.785537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d41f19494b47d0935c496f603d2960'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:34.790677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c532537-b8bc-41a3-9c01-084728939807 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:34.802638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd8935-1ea0-49ae-a73a-34581f830f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:34.803452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8935-1ea0-49ae-a73a-34581f830f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:34.803850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8935-1ea0-49ae-a73a-34581f830f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:34.804802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8935-1ea0-49ae-a73a-34581f830f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:34.805248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd8935-1ea0-49ae-a73a-34581f830f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:34.808240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd8935-1ea0-49ae-a73a-34581f830f04 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:34.825629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8e4f4df6-e900-4a95-b0d5-7eddd488e577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d41f19494b47d0935c496f603d2960'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:34.832540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1163/2309] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:34.837819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd8935-1ea0-49ae-a73a-34581f830f04 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:34.839030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1147/2310] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:34 2022] DELETE /identity/v3/users/f8d41f19494b47d0935c496f603d2960 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:51:34.842745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e8cd8935-1ea0-49ae-a73a-34581f830f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:34.877934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd8935-1ea0-49ae-a73a-34581f830f04 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:35.050742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842d938-3597-4cf3-9539-0696ebcbb8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:35.051846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842d938-3597-4cf3-9539-0696ebcbb8eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:35.052131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842d938-3597-4cf3-9539-0696ebcbb8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:35.052470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842d938-3597-4cf3-9539-0696ebcbb8eb admin admin] PATH_INFO: `/v3/projects/f18b43c128844ad887e5fb11908a1841/users/2bf5d49e598f4d47be906017890fb82a/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:35.071905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842d938-3597-4cf3-9539-0696ebcbb8eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18b43c128844ad887e5fb11908a1841, user_id=2bf5d49e598f4d47be906017890fb82a, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:35.074879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842d938-3597-4cf3-9539-0696ebcbb8eb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:35.197921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1164/2311] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:34 2022] PUT /identity/v3/projects/f18b43c128844ad887e5fb11908a1841/users/2bf5d49e598f4d47be906017890fb82a/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:35.209238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f883e77-fa81-4fde-84bc-9527d8b4a317 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:35.215029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f883e77-fa81-4fde-84bc-9527d8b4a317 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:35.215029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f883e77-fa81-4fde-84bc-9527d8b4a317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:35.266733 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-2f883e77-fa81-4fde-84bc-9527d8b4a317 None None] MFA Rules not processed for user `2bf5d49e598f4d47be906017890fb82a`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:35.268570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-2f883e77-fa81-4fde-84bc-9527d8b4a317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:35.408902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1148/2312] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:35.422042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842d938-3597-4cf3-9539-0696ebcbb8eb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:35.424847 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1842d938-3597-4cf3-9539-0696ebcbb8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:35.495409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842d938-3597-4cf3-9539-0696ebcbb8eb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:35.504021 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a20195-cfcd-4f5c-91ed-300e02533e21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:35.506237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a20195-cfcd-4f5c-91ed-300e02533e21 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:35.506760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a20195-cfcd-4f5c-91ed-300e02533e21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:35.507484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a20195-cfcd-4f5c-91ed-300e02533e21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:35.519565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a20195-cfcd-4f5c-91ed-300e02533e21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:35.522971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a20195-cfcd-4f5c-91ed-300e02533e21 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:35.534387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1165/2313] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:35.561936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f883e77-fa81-4fde-84bc-9527d8b4a317 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:35.602776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a20195-cfcd-4f5c-91ed-300e02533e21 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:35.608861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-95a20195-cfcd-4f5c-91ed-300e02533e21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:35.678002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f883e77-fa81-4fde-84bc-9527d8b4a317 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:35.690795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:35.692114 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:35.692544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:35.692914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] PATH_INFO: `/v3/users/2bf5d49e598f4d47be906017890fb82a` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:35.703029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf5d49e598f4d47be906017890fb82a)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:35.708367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:35.775352 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a20195-cfcd-4f5c-91ed-300e02533e21 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:35.781677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:35.782423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:35.782694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:35.793804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] PATH_INFO: `/v3/projects/cea423049d40432abc2b3dc512dc3016` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:35.793804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea423049d40432abc2b3dc512dc3016)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:35.797528 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:35.844666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:35.983523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea423049d40432abc2b3dc512dc3016', 'request_id': 'req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:35.998054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea423049d40432abc2b3dc512dc3016', 'request_id': 'req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:36.005659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf5d49e598f4d47be906017890fb82a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:36.128942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea423049d40432abc2b3dc512dc3016', 'request_id': 'req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:36.147993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] The token cache is being invalidate because project cea423049d40432abc2b3dc512dc3016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:36.148285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:36.149016 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d5176a1e-bdaa-49fb-a7e4-d27eb74b42b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:36.151144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1166/2314] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:35 2022] DELETE /identity/v3/projects/cea423049d40432abc2b3dc512dc3016 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:36.155594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeace3-4242-4b67-be08-aba5ccf1d118 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:36.155947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeace3-4242-4b67-be08-aba5ccf1d118 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:36.156284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeace3-4242-4b67-be08-aba5ccf1d118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:36.163672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:36.185643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-66aeace3-4242-4b67-be08-aba5ccf1d118 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:36.187401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-66aeace3-4242-4b67-be08-aba5ccf1d118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:36.230035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf5d49e598f4d47be906017890fb82a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:36.230953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf5d49e598f4d47be906017890fb82a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:36.241228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf5d49e598f4d47be906017890fb82a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:36.265197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c9e4219a-3dc4-4cd6-b843-6122827a816a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf5d49e598f4d47be906017890fb82a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:36.279503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1167/2315] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:36.287776 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1149/2316] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:35 2022] DELETE /identity/v3/users/2bf5d49e598f4d47be906017890fb82a => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:36.291294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb82db2-4e64-4d51-8c74-7056650f467f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:36.291896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fddd0f-57c1-4cc9-921c-d3118c0d6d45 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:36.292184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fddd0f-57c1-4cc9-921c-d3118c0d6d45 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:36.292454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb82db2-4e64-4d51-8c74-7056650f467f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:36.292610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fddd0f-57c1-4cc9-921c-d3118c0d6d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:36.292750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb82db2-4e64-4d51-8c74-7056650f467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:36.342276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8fb82db2-4e64-4d51-8c74-7056650f467f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:36.344219 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8fb82db2-4e64-4d51-8c74-7056650f467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:36.367631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-c4fddd0f-57c1-4cc9-921c-d3118c0d6d45 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:36.369236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c4fddd0f-57c1-4cc9-921c-d3118c0d6d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:36.440617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1168/2317] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:36.445320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb82db2-4e64-4d51-8c74-7056650f467f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:36.455729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1150/2318] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:36.457254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb82db2-4e64-4d51-8c74-7056650f467f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:36.463617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04044f3-dd61-4048-ab6a-9049a10f87f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:36.464334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04044f3-dd61-4048-ab6a-9049a10f87f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:36.464878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04044f3-dd61-4048-ab6a-9049a10f87f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:36.465154 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04044f3-dd61-4048-ab6a-9049a10f87f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:36.465880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04044f3-dd61-4048-ab6a-9049a10f87f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:51:36.466189 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04044f3-dd61-4048-ab6a-9049a10f87f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:36.466323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fddd0f-57c1-4cc9-921c-d3118c0d6d45 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:36.468033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04044f3-dd61-4048-ab6a-9049a10f87f1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:36.469520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c4fddd0f-57c1-4cc9-921c-d3118c0d6d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:36.508756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1169/2319] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:51:36.516840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04044f3-dd61-4048-ab6a-9049a10f87f1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:36.544651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04044f3-dd61-4048-ab6a-9049a10f87f1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:36.556005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42b3da-db83-42dc-a7bf-12c844bae264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:36.556631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42b3da-db83-42dc-a7bf-12c844bae264 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:36.556913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42b3da-db83-42dc-a7bf-12c844bae264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:36.557195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42b3da-db83-42dc-a7bf-12c844bae264 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:36.558171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42b3da-db83-42dc-a7bf-12c844bae264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:36.562056 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42b3da-db83-42dc-a7bf-12c844bae264 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:36.581348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fddd0f-57c1-4cc9-921c-d3118c0d6d45 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:36.585906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55798091-f574-4bdb-9141-5c8200b4a580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:36.586523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55798091-f574-4bdb-9141-5c8200b4a580 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:36.586813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55798091-f574-4bdb-9141-5c8200b4a580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:36.587083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55798091-f574-4bdb-9141-5c8200b4a580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:36.597644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55798091-f574-4bdb-9141-5c8200b4a580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:36.599777 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55798091-f574-4bdb-9141-5c8200b4a580 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:36.610925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1151/2320] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:51:36.710709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1170/2321] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:36 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:36.714952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55798091-f574-4bdb-9141-5c8200b4a580 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:36.725877 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55798091-f574-4bdb-9141-5c8200b4a580 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:36.801708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725623e1-034f-4f3a-9820-646c9cf4cc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:36.801708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725623e1-034f-4f3a-9820-646c9cf4cc02 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:36.801708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725623e1-034f-4f3a-9820-646c9cf4cc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:36.801708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725623e1-034f-4f3a-9820-646c9cf4cc02 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:36.802341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725623e1-034f-4f3a-9820-646c9cf4cc02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:36.802783 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725623e1-034f-4f3a-9820-646c9cf4cc02 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:36.938150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1152/2322] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:36 2022] POST /identity/v3/users => generated 290 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:36.953455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42b3da-db83-42dc-a7bf-12c844bae264 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:36.970225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42b3da-db83-42dc-a7bf-12c844bae264 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:36.976064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18745cee-1fa6-4fcb-9cd8-5c97bc2d9841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:36.976681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18745cee-1fa6-4fcb-9cd8-5c97bc2d9841 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:36.976990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18745cee-1fa6-4fcb-9cd8-5c97bc2d9841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:36.977298 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18745cee-1fa6-4fcb-9cd8-5c97bc2d9841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:36.977998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18745cee-1fa6-4fcb-9cd8-5c97bc2d9841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:36.979618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18745cee-1fa6-4fcb-9cd8-5c97bc2d9841 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:36.992213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1171/2323] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:37.002645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725623e1-034f-4f3a-9820-646c9cf4cc02 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:37.007769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725623e1-034f-4f3a-9820-646c9cf4cc02 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:37.013070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c10b1-93fa-44b9-8c41-a4b692480628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:37.013695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c10b1-93fa-44b9-8c41-a4b692480628 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:37.014241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c10b1-93fa-44b9-8c41-a4b692480628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:37.014518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c10b1-93fa-44b9-8c41-a4b692480628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:37.015172 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c10b1-93fa-44b9-8c41-a4b692480628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:37.016829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c10b1-93fa-44b9-8c41-a4b692480628 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:37.030671 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1153/2324] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:37.037646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18745cee-1fa6-4fcb-9cd8-5c97bc2d9841 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:37.048358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18745cee-1fa6-4fcb-9cd8-5c97bc2d9841 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:37.055238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274c408-f838-4d7b-96a9-daad503cd753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:37.055585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274c408-f838-4d7b-96a9-daad503cd753 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:37.055878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274c408-f838-4d7b-96a9-daad503cd753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:37.056970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274c408-f838-4d7b-96a9-daad503cd753 admin admin] PATH_INFO: `/v3/projects/10e2169e3405486faf19ab6b70c74598/users/a2e842f8b1a0497eafd4d39ea06e47aa/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:37.074882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274c408-f838-4d7b-96a9-daad503cd753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e2169e3405486faf19ab6b70c74598, user_id=a2e842f8b1a0497eafd4d39ea06e47aa, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:37.077667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274c408-f838-4d7b-96a9-daad503cd753 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:37.193955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:37.232413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c10b1-93fa-44b9-8c41-a4b692480628 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:37.234452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c35c10b1-93fa-44b9-8c41-a4b692480628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:37.236612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1172/2325] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:37 2022] PUT /identity/v3/projects/10e2169e3405486faf19ab6b70c74598/users/a2e842f8b1a0497eafd4d39ea06e47aa/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:37.250373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d5ff4-b8c9-42c2-9fa5-38487221e461 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:37.250667 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d5ff4-b8c9-42c2-9fa5-38487221e461 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:37.250985 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d5ff4-b8c9-42c2-9fa5-38487221e461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:37.251754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c10b1-93fa-44b9-8c41-a4b692480628 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:37.309517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3e4d5ff4-b8c9-42c2-9fa5-38487221e461 None None] MFA Rules not processed for user `a2e842f8b1a0497eafd4d39ea06e47aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:37.310900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3e4d5ff4-b8c9-42c2-9fa5-38487221e461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:37.363057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:37.363704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:37.364213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:37.365809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] PATH_INFO: `/v3/projects/f18b43c128844ad887e5fb11908a1841` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:37.379184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18b43c128844ad887e5fb11908a1841)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:37.385587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:37.444217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1173/2326] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:37.446996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82eeb52-b2e4-4ec8-992e-3be47add9352 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:37.447697 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82eeb52-b2e4-4ec8-992e-3be47add9352 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:37.449072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82eeb52-b2e4-4ec8-992e-3be47add9352 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:37.453633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1174/2327] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:51:37 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:51:37.454623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82eeb52-b2e4-4ec8-992e-3be47add9352 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:37.478840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82eeb52-b2e4-4ec8-992e-3be47add9352 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:37.485043 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd0a10-6cc3-44c4-b3ac-4db1dd156888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:37.485762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0a10-6cc3-44c4-b3ac-4db1dd156888 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:37.486122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0a10-6cc3-44c4-b3ac-4db1dd156888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:37.486360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0a10-6cc3-44c4-b3ac-4db1dd156888 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:37.487002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd0a10-6cc3-44c4-b3ac-4db1dd156888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:37.488997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd0a10-6cc3-44c4-b3ac-4db1dd156888 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:37.550496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18b43c128844ad887e5fb11908a1841', 'request_id': 'req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:37.641157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1175/2328] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:37 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:37.642374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff5b17-5ea2-4c75-8bd6-e87f8a03a1f6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:37.642675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff5b17-5ea2-4c75-8bd6-e87f8a03a1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:37.642947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff5b17-5ea2-4c75-8bd6-e87f8a03a1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:37.663180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d5ff5b17-5ea2-4c75-8bd6-e87f8a03a1f6 None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:37.664415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d5ff5b17-5ea2-4c75-8bd6-e87f8a03a1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:37.667273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18b43c128844ad887e5fb11908a1841', 'request_id': 'req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:37.676335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18b43c128844ad887e5fb11908a1841', 'request_id': 'req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:37.707300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] The token cache is being invalidate because project f18b43c128844ad887e5fb11908a1841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:37.708706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:37.710378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:37.721412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1154/2329] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:37 2022] DELETE /identity/v3/projects/f18b43c128844ad887e5fb11908a1841 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:37.721953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:37.726871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:37.799223 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1176/2330] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:51:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:37.803736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77c84f-3cf4-47e9-8b31-bee5e528dda0 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:37.804607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77c84f-3cf4-47e9-8b31-bee5e528dda0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:37.804896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77c84f-3cf4-47e9-8b31-bee5e528dda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:37.836317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0e77c84f-3cf4-47e9-8b31-bee5e528dda0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:37.843769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0e77c84f-3cf4-47e9-8b31-bee5e528dda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:37.898137 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce1e89-1dbc-4fb6-a820-09ed1c70ab55 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:37.937770 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1177/2331] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:37.938718 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77c84f-3cf4-47e9-8b31-bee5e528dda0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:37.940107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0e77c84f-3cf4-47e9-8b31-bee5e528dda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:37.966989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fcf03-e6b9-4f2c-811a-5b5a8ef7d250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:37.967754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fcf03-e6b9-4f2c-811a-5b5a8ef7d250 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:37.968101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fcf03-e6b9-4f2c-811a-5b5a8ef7d250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:37.968479 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fcf03-e6b9-4f2c-811a-5b5a8ef7d250 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:37.969480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fcf03-e6b9-4f2c-811a-5b5a8ef7d250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:37.973152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fcf03-e6b9-4f2c-811a-5b5a8ef7d250 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:38.041572 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77c84f-3cf4-47e9-8b31-bee5e528dda0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:38.044914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:38.045507 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a111a-7a7c-4a03-95da-83d0665a691a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab54ee52004746eaadb5c2970e59afc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:38.046267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a111a-7a7c-4a03-95da-83d0665a691a service glance] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.046595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a111a-7a7c-4a03-95da-83d0665a691a service glance] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.047107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a111a-7a7c-4a03-95da-83d0665a691a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.069025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a111a-7a7c-4a03-95da-83d0665a691a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:38.069025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a111a-7a7c-4a03-95da-83d0665a691a service glance] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:38.082620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1178/2332] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:38.085074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78ef99-2d41-4077-b358-0079959bfc4e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.085442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78ef99-2d41-4077-b358-0079959bfc4e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.085810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78ef99-2d41-4077-b358-0079959bfc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.103794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1155/2333] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:37 2022] POST /identity/v3/users => generated 290 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:38.108579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-dc78ef99-2d41-4077-b358-0079959bfc4e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:38.110437 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fcf03-e6b9-4f2c-811a-5b5a8ef7d250 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:38.118807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dc78ef99-2d41-4077-b358-0079959bfc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:38.139135 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fcf03-e6b9-4f2c-811a-5b5a8ef7d250 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:38.151599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1179/2334] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:38.152068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64876f1-784a-4c36-bb86-646196f5dd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:38.152853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64876f1-784a-4c36-bb86-646196f5dd9a admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.153134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64876f1-784a-4c36-bb86-646196f5dd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.153564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64876f1-784a-4c36-bb86-646196f5dd9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.154345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64876f1-784a-4c36-bb86-646196f5dd9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:38.156901 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78ef99-2d41-4077-b358-0079959bfc4e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:38.177342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64876f1-784a-4c36-bb86-646196f5dd9a admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:38.186269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78ef99-2d41-4077-b358-0079959bfc4e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:38.197364 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1156/2335] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:51:38.198027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64c740-7106-480f-bbb1-fa77b33668c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:38.199026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64c740-7106-480f-bbb1-fa77b33668c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.199484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64c740-7106-480f-bbb1-fa77b33668c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.201850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64876f1-784a-4c36-bb86-646196f5dd9a admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:38.202375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64c740-7106-480f-bbb1-fa77b33668c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.203175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64c740-7106-480f-bbb1-fa77b33668c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:51:38.203548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64c740-7106-480f-bbb1-fa77b33668c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:38.205571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64c740-7106-480f-bbb1-fa77b33668c4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:38.220950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64876f1-784a-4c36-bb86-646196f5dd9a admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:38.226553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb30d19-3c79-4025-9673-584a894cd35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:38.227587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb30d19-3c79-4025-9673-584a894cd35d admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.227947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb30d19-3c79-4025-9673-584a894cd35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.228630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb30d19-3c79-4025-9673-584a894cd35d admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.229346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb30d19-3c79-4025-9673-584a894cd35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:38.229797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1180/2336] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:51:38.231941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb30d19-3c79-4025-9673-584a894cd35d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:38.234246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64c740-7106-480f-bbb1-fa77b33668c4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:38.245802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1157/2337] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:51:38.246237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64c740-7106-480f-bbb1-fa77b33668c4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:38.248738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb30d19-3c79-4025-9673-584a894cd35d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:38.251821 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dc97b-889b-4e92-81c1-17ffe862e132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:38.253123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dc97b-889b-4e92-81c1-17ffe862e132 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.253123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dc97b-889b-4e92-81c1-17ffe862e132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.253123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dc97b-889b-4e92-81c1-17ffe862e132 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.253783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3dc97b-889b-4e92-81c1-17ffe862e132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:38.257068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3dc97b-889b-4e92-81c1-17ffe862e132 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:38.260602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb30d19-3c79-4025-9673-584a894cd35d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:38.267075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014280ec-ac6f-4ff0-a3b9-aebebd6070fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:38.267816 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014280ec-ac6f-4ff0-a3b9-aebebd6070fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.268157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014280ec-ac6f-4ff0-a3b9-aebebd6070fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.268521 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014280ec-ac6f-4ff0-a3b9-aebebd6070fc admin admin] PATH_INFO: `/v3/projects/0287c5d55f564217a41ea6ff52d32d95/users/e687269edc5649098f4124444ac52fdf/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.284330 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014280ec-ac6f-4ff0-a3b9-aebebd6070fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0287c5d55f564217a41ea6ff52d32d95, user_id=e687269edc5649098f4124444ac52fdf, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:38.287270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014280ec-ac6f-4ff0-a3b9-aebebd6070fc admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:38.385076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1181/2338] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:38 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:38.386750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dc97b-889b-4e92-81c1-17ffe862e132 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:38.437848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1158/2339] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:38 2022] PUT /identity/v3/projects/0287c5d55f564217a41ea6ff52d32d95/users/e687269edc5649098f4124444ac52fdf/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:51:38.451111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b766d19-e22a-4929-be9c-069d2b5e3489 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.451494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b766d19-e22a-4929-be9c-069d2b5e3489 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.451867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b766d19-e22a-4929-be9c-069d2b5e3489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.484460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dc97b-889b-4e92-81c1-17ffe862e132 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:38.493939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-5b766d19-e22a-4929-be9c-069d2b5e3489 None None] MFA Rules not processed for user `e687269edc5649098f4124444ac52fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:38.495292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5b766d19-e22a-4929-be9c-069d2b5e3489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:38.573868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b774b-1cbf-4ed2-a38b-c3ab80277933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:38.581974 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b774b-1cbf-4ed2-a38b-c3ab80277933 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.583993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b774b-1cbf-4ed2-a38b-c3ab80277933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.584299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b774b-1cbf-4ed2-a38b-c3ab80277933 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.585097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b774b-1cbf-4ed2-a38b-c3ab80277933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:38.587502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b774b-1cbf-4ed2-a38b-c3ab80277933 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:38.603729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1159/2340] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:38.608684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eef182-4fe5-4905-822d-ad322fc545d7 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.608968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eef182-4fe5-4905-822d-ad322fc545d7 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.609312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eef182-4fe5-4905-822d-ad322fc545d7 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.610842 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1160/2341] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:51:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:51:38.611449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eef182-4fe5-4905-822d-ad322fc545d7 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:38.632387 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eef182-4fe5-4905-822d-ad322fc545d7 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:38.636920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fc1aa-73b4-464b-9677-93c2862f9348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:38.637518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fc1aa-73b4-464b-9677-93c2862f9348 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.637809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fc1aa-73b4-464b-9677-93c2862f9348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.638079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fc1aa-73b4-464b-9677-93c2862f9348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.638813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fc1aa-73b4-464b-9677-93c2862f9348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:38.640716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fc1aa-73b4-464b-9677-93c2862f9348 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:38.730683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1182/2342] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:38 2022] POST /identity/v3/users => generated 292 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:51:38.733028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c1dd-325e-4cf2-9869-440e5d66ff66 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.733367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c1dd-325e-4cf2-9869-440e5d66ff66 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.733503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c1dd-325e-4cf2-9869-440e5d66ff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.763123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f142c1dd-325e-4cf2-9869-440e5d66ff66 None None] MFA Rules not processed for user `6336f0c469b84db78a90cf8435c7dbe4`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:38.764406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f142c1dd-325e-4cf2-9869-440e5d66ff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:38.814824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:38.824554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1161/2343] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:38 2022] POST /identity/v3/projects => generated 379 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:38.824554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fc1aa-73b4-464b-9677-93c2862f9348 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:38.872253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1183/2344] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:51:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:38.878998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c1dd-325e-4cf2-9869-440e5d66ff66 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:38.891884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c1dd-325e-4cf2-9869-440e5d66ff66 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:38.917749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fc1aa-73b4-464b-9677-93c2862f9348 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:38.923748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef834a3-39c5-48ea-8037-6be80c5d0aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:38.924590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef834a3-39c5-48ea-8037-6be80c5d0aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.924867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef834a3-39c5-48ea-8037-6be80c5d0aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.925209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef834a3-39c5-48ea-8037-6be80c5d0aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.925959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef834a3-39c5-48ea-8037-6be80c5d0aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:38.927995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef834a3-39c5-48ea-8037-6be80c5d0aa7 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:38.941762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1162/2345] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:38.944634 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef834a3-39c5-48ea-8037-6be80c5d0aa7 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:38.951562 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bef834a3-39c5-48ea-8037-6be80c5d0aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:38.989791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947e296-00d7-4dea-839a-16b4957a5384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:38.991472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947e296-00d7-4dea-839a-16b4957a5384 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:38.995939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947e296-00d7-4dea-839a-16b4957a5384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:38.999764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947e296-00d7-4dea-839a-16b4957a5384 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:38.999764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947e296-00d7-4dea-839a-16b4957a5384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:38.999764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947e296-00d7-4dea-839a-16b4957a5384 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.057678 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef834a3-39c5-48ea-8037-6be80c5d0aa7 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:39.062860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f9a39-2ca9-44ad-926a-c82ad6a39249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:39.063555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f9a39-2ca9-44ad-926a-c82ad6a39249 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.064587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f9a39-2ca9-44ad-926a-c82ad6a39249 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.064888 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f9a39-2ca9-44ad-926a-c82ad6a39249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.073790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f9a39-2ca9-44ad-926a-c82ad6a39249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:39.075719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f9a39-2ca9-44ad-926a-c82ad6a39249 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.086517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1163/2346] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:39.087430 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f9a39-2ca9-44ad-926a-c82ad6a39249 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:39.102432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f9a39-2ca9-44ad-926a-c82ad6a39249 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:39.107313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe667026-9036-44c8-87e2-14afaf00856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:39.108022 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe667026-9036-44c8-87e2-14afaf00856e admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.108361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe667026-9036-44c8-87e2-14afaf00856e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.108695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe667026-9036-44c8-87e2-14afaf00856e admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.109417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe667026-9036-44c8-87e2-14afaf00856e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:39.111207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe667026-9036-44c8-87e2-14afaf00856e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.132319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1184/2347] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:38 2022] POST /identity/v3/users => generated 296 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:39.133033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1164/2348] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:39.133123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947e296-00d7-4dea-839a-16b4957a5384 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:39.134924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe667026-9036-44c8-87e2-14afaf00856e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:39.146469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe667026-9036-44c8-87e2-14afaf00856e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:39.151756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6577578-9e94-450a-977a-74a1b763b84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:39.152092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6577578-9e94-450a-977a-74a1b763b84d admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.152369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6577578-9e94-450a-977a-74a1b763b84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.152635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6577578-9e94-450a-977a-74a1b763b84d admin admin] PATH_INFO: `/v3/projects/0cdb8140dca34727ab99c95efc7acb81/users/fb627e22492140858b12dcf4c0a70821/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.157164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947e296-00d7-4dea-839a-16b4957a5384 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:39.164487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b27d3-68b4-42c3-88af-a5d462edc15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:39.165363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b27d3-68b4-42c3-88af-a5d462edc15c admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.165860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b27d3-68b4-42c3-88af-a5d462edc15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.166135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b27d3-68b4-42c3-88af-a5d462edc15c admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.166788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b27d3-68b4-42c3-88af-a5d462edc15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:39.168790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b27d3-68b4-42c3-88af-a5d462edc15c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.169659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6577578-9e94-450a-977a-74a1b763b84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cdb8140dca34727ab99c95efc7acb81, user_id=fb627e22492140858b12dcf4c0a70821, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:39.173113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6577578-9e94-450a-977a-74a1b763b84d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.185784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1185/2349] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:39.189283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b27d3-68b4-42c3-88af-a5d462edc15c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:39.199910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b27d3-68b4-42c3-88af-a5d462edc15c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:39.205321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66080115-4ee8-4831-bd8c-cc800603818b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:39.206257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66080115-4ee8-4831-bd8c-cc800603818b admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.206606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66080115-4ee8-4831-bd8c-cc800603818b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.206945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66080115-4ee8-4831-bd8c-cc800603818b admin admin] PATH_INFO: `/v3/projects/360f6e55040a42deb6dde6d90416e4e5/users/5a57c2ceeb5d40e3934647f2075a27c3/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.243262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66080115-4ee8-4831-bd8c-cc800603818b admin admin] RBAC: Authorizing `identity:create_grant(project_id=360f6e55040a42deb6dde6d90416e4e5, user_id=5a57c2ceeb5d40e3934647f2075a27c3, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:39.246983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66080115-4ee8-4831-bd8c-cc800603818b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.297060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1165/2350] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:39 2022] PUT /identity/v3/projects/0cdb8140dca34727ab99c95efc7acb81/users/fb627e22492140858b12dcf4c0a70821/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:39.298371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed81102-f472-4cc5-a120-01013ac6c636 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.298675 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed81102-f472-4cc5-a120-01013ac6c636 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.298923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed81102-f472-4cc5-a120-01013ac6c636 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.300332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1166/2351] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:51:39.301794 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3291-7510-4a5a-90fc-89c90801c672 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.302551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3291-7510-4a5a-90fc-89c90801c672 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.302551 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3291-7510-4a5a-90fc-89c90801c672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.322654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-f2bc3291-7510-4a5a-90fc-89c90801c672 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:39.323720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f2bc3291-7510-4a5a-90fc-89c90801c672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:39.357620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1186/2352] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:39 2022] PUT /identity/v3/projects/360f6e55040a42deb6dde6d90416e4e5/users/5a57c2ceeb5d40e3934647f2075a27c3/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:39.359189 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b11d46-6819-4482-b12a-55d47e384ec1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.359664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b11d46-6819-4482-b12a-55d47e384ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.360164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b11d46-6819-4482-b12a-55d47e384ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.382707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b7b11d46-6819-4482-b12a-55d47e384ec1 None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:39.384180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b7b11d46-6819-4482-b12a-55d47e384ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:39.398292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1167/2353] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:39.403759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3291-7510-4a5a-90fc-89c90801c672 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:39.415875 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3291-7510-4a5a-90fc-89c90801c672 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:39.465688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1187/2354] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:51:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:39.466630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b11d46-6819-4482-b12a-55d47e384ec1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:39.482542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab00814-6ca6-47c3-b5ef-9405f9bb79d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:39.483205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab00814-6ca6-47c3-b5ef-9405f9bb79d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.483774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab00814-6ca6-47c3-b5ef-9405f9bb79d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.484064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab00814-6ca6-47c3-b5ef-9405f9bb79d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.484722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab00814-6ca6-47c3-b5ef-9405f9bb79d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:39.486517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab00814-6ca6-47c3-b5ef-9405f9bb79d2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.505813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1168/2355] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:39.507100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab00814-6ca6-47c3-b5ef-9405f9bb79d2 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:39.548198 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b11d46-6819-4482-b12a-55d47e384ec1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:39.553042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5473144-ef7b-4d46-a7bd-62bc2ddd5b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:39.553763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5473144-ef7b-4d46-a7bd-62bc2ddd5b63 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.554115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5473144-ef7b-4d46-a7bd-62bc2ddd5b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.554459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5473144-ef7b-4d46-a7bd-62bc2ddd5b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.566096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5473144-ef7b-4d46-a7bd-62bc2ddd5b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:39.572010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5473144-ef7b-4d46-a7bd-62bc2ddd5b63 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.589672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1188/2356] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:39.592626 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5473144-ef7b-4d46-a7bd-62bc2ddd5b63 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:39.593819 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab00814-6ca6-47c3-b5ef-9405f9bb79d2 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:39.608177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e9113-6b92-4ed4-9a46-57b02cdb3bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:39.608177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e9113-6b92-4ed4-9a46-57b02cdb3bfc service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.608177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e9113-6b92-4ed4-9a46-57b02cdb3bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.608177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e9113-6b92-4ed4-9a46-57b02cdb3bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.612896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5473144-ef7b-4d46-a7bd-62bc2ddd5b63 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:39.626361 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b52915-7217-49f8-a0fe-e4e041b197c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:39.627092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b52915-7217-49f8-a0fe-e4e041b197c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.627383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b52915-7217-49f8-a0fe-e4e041b197c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.628030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b52915-7217-49f8-a0fe-e4e041b197c5 admin admin] PATH_INFO: `/v3/projects/360f6e55040a42deb6dde6d90416e4e5/users/5a57c2ceeb5d40e3934647f2075a27c3/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.629035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e9113-6b92-4ed4-9a46-57b02cdb3bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:39.633196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e9113-6b92-4ed4-9a46-57b02cdb3bfc service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.638267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b52915-7217-49f8-a0fe-e4e041b197c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=360f6e55040a42deb6dde6d90416e4e5, user_id=5a57c2ceeb5d40e3934647f2075a27c3, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:39.644712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b52915-7217-49f8-a0fe-e4e041b197c5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.647073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1169/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:39.659376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1189/2358] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:39 2022] PUT /identity/v3/projects/360f6e55040a42deb6dde6d90416e4e5/users/5a57c2ceeb5d40e3934647f2075a27c3/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:39.675821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7d15f-a385-4ad9-b732-d8c38aa212ac None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.676110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7d15f-a385-4ad9-b732-d8c38aa212ac None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.676446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7d15f-a385-4ad9-b732-d8c38aa212ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.709140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-83f7d15f-a385-4ad9-b732-d8c38aa212ac None None] MFA Rules not processed for user `5a57c2ceeb5d40e3934647f2075a27c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:39.710485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-83f7d15f-a385-4ad9-b732-d8c38aa212ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:39.771200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:39.792962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1170/2359] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:39.805928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:39.807237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b52915-7217-49f8-a0fe-e4e041b197c5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:39.809567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-52b52915-7217-49f8-a0fe-e4e041b197c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:39.874857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b52915-7217-49f8-a0fe-e4e041b197c5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:39.879099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcacc20-0a29-4246-b345-53564c8eed00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:39.879794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcacc20-0a29-4246-b345-53564c8eed00 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:39.880154 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcacc20-0a29-4246-b345-53564c8eed00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:39.880940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcacc20-0a29-4246-b345-53564c8eed00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:39.890789 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcacc20-0a29-4246-b345-53564c8eed00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:39.892552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcacc20-0a29-4246-b345-53564c8eed00 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:39.904014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1190/2360] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:40.057539 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7d15f-a385-4ad9-b732-d8c38aa212ac None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:40.076506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7d15f-a385-4ad9-b732-d8c38aa212ac None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:40.082019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5d82a-ac8c-4d24-a3f7-0291818d3345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:40.083344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5d82a-ac8c-4d24-a3f7-0291818d3345 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:40.084065 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5d82a-ac8c-4d24-a3f7-0291818d3345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:40.084748 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5d82a-ac8c-4d24-a3f7-0291818d3345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:40.088067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b5c5d82a-ac8c-4d24-a3f7-0291818d3345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:40.181389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5d82a-ac8c-4d24-a3f7-0291818d3345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:40.187381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5d82a-ac8c-4d24-a3f7-0291818d3345 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:40.195109 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1171/2361] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:41.514092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:42.005370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:43.207473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:43.902278 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:45.171844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:45.220137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcacc20-0a29-4246-b345-53564c8eed00 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:45.221869 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9dcacc20-0a29-4246-b345-53564c8eed00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:45.306695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcacc20-0a29-4246-b345-53564c8eed00 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:45.318347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:45.319443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:45.320346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:45.320684 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] PATH_INFO: `/v3/users/29375fdec6fd4a93ab440a11d8aa6fc4` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:45.322424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29375fdec6fd4a93ab440a11d8aa6fc4)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:45.326229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:45.583268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29375fdec6fd4a93ab440a11d8aa6fc4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:45.633910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29375fdec6fd4a93ab440a11d8aa6fc4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:45.728609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29375fdec6fd4a93ab440a11d8aa6fc4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:45.729846 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29375fdec6fd4a93ab440a11d8aa6fc4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:45.748543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29375fdec6fd4a93ab440a11d8aa6fc4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:45.759456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1191/2362] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:45 2022] DELETE /identity/v3/users/29375fdec6fd4a93ab440a11d8aa6fc4 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:45.762764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d05a7-d347-4ab1-817a-611f8442dbdc None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:45.763132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d05a7-d347-4ab1-817a-611f8442dbdc None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:45.763488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d05a7-d347-4ab1-817a-611f8442dbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:45.792216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-766d05a7-d347-4ab1-817a-611f8442dbdc None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:45.798696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-766d05a7-d347-4ab1-817a-611f8442dbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:45.914365 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1172/2363] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:45.944649 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:45.946179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:45.999545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:46.079475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3f244-dcff-49f3-bd87-a7262bae6491 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:46.084913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715657f5-3500-4a22-8235-cf867af30620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:46.086180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715657f5-3500-4a22-8235-cf867af30620 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:46.086618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715657f5-3500-4a22-8235-cf867af30620 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:46.087172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715657f5-3500-4a22-8235-cf867af30620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:46.120170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715657f5-3500-4a22-8235-cf867af30620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:46.121758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715657f5-3500-4a22-8235-cf867af30620 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:46.135436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1192/2364] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:46.890621 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d05a7-d347-4ab1-817a-611f8442dbdc None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:46.894535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-766d05a7-d347-4ab1-817a-611f8442dbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:46.927476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d05a7-d347-4ab1-817a-611f8442dbdc None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:46.964260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:46.965003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:46.965363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:46.965768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] PATH_INFO: `/v3/users/a2e842f8b1a0497eafd4d39ea06e47aa` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:46.971140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e842f8b1a0497eafd4d39ea06e47aa)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:46.971140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:47.143866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715657f5-3500-4a22-8235-cf867af30620 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:47.146207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-715657f5-3500-4a22-8235-cf867af30620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:47.164583 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715657f5-3500-4a22-8235-cf867af30620 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:47.171131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:47.171712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:47.172026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:47.172304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] PATH_INFO: `/v3/projects/d1f09cb3ec0147b39302fd2df9e89a5b` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:47.173467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f09cb3ec0147b39302fd2df9e89a5b)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:47.175418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:47.306382 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e842f8b1a0497eafd4d39ea06e47aa'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:47.357628 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f09cb3ec0147b39302fd2df9e89a5b', 'request_id': 'req-273f1abf-c130-485f-9147-c4f6f3f55dc9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:47.390503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f09cb3ec0147b39302fd2df9e89a5b', 'request_id': 'req-273f1abf-c130-485f-9147-c4f6f3f55dc9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:47.431160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e842f8b1a0497eafd4d39ea06e47aa'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:47.433083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e842f8b1a0497eafd4d39ea06e47aa'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:47.446734 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e842f8b1a0497eafd4d39ea06e47aa'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:47.472096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-30d84be1-1612-4d8f-aa90-755f8d9350fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e842f8b1a0497eafd4d39ea06e47aa'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:47.487489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1173/2365] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:46 2022] DELETE /identity/v3/users/a2e842f8b1a0497eafd4d39ea06e47aa => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:47.492852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49ab35-749c-4f9c-99dd-c8c42124ecfa None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:47.493351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49ab35-749c-4f9c-99dd-c8c42124ecfa None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:47.493983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49ab35-749c-4f9c-99dd-c8c42124ecfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:47.509817 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f09cb3ec0147b39302fd2df9e89a5b', 'request_id': 'req-273f1abf-c130-485f-9147-c4f6f3f55dc9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:47.528559 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] The token cache is being invalidate because project d1f09cb3ec0147b39302fd2df9e89a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:47.529090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:47.530148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:47.541802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-bd49ab35-749c-4f9c-99dd-c8c42124ecfa None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:47.543906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bd49ab35-749c-4f9c-99dd-c8c42124ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:47.556553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1193/2366] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:47 2022] DELETE /identity/v3/projects/d1f09cb3ec0147b39302fd2df9e89a5b => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:47.688592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:47.721457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1174/2367] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:47.774880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:47.796051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:47.859837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27b432-fc35-4787-8e69-eb49fd606bcf None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:47.860170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27b432-fc35-4787-8e69-eb49fd606bcf None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:47.860455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27b432-fc35-4787-8e69-eb49fd606bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:47.925691 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f1abf-c130-485f-9147-c4f6f3f55dc9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:47.945515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-de27b432-fc35-4787-8e69-eb49fd606bcf None None] MFA Rules not processed for user `fb627e22492140858b12dcf4c0a70821`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:47.948855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-de27b432-fc35-4787-8e69-eb49fd606bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:48.041823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350cb68-86f5-494a-99bc-95d357418d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:48.042563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350cb68-86f5-494a-99bc-95d357418d07 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:48.042944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350cb68-86f5-494a-99bc-95d357418d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:48.043413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350cb68-86f5-494a-99bc-95d357418d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:48.070936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350cb68-86f5-494a-99bc-95d357418d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:48.072609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350cb68-86f5-494a-99bc-95d357418d07 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:48.088557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1175/2368] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:48.093487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27b432-fc35-4787-8e69-eb49fd606bcf None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:48.093633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1194/2369] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:48.095240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-de27b432-fc35-4787-8e69-eb49fd606bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:48.113938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27b432-fc35-4787-8e69-eb49fd606bcf None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:48.119711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb26e2-97d4-42a2-8ede-022f6bb6900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:48.120600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb26e2-97d4-42a2-8ede-022f6bb6900d admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:48.120891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb26e2-97d4-42a2-8ede-022f6bb6900d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:48.121173 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb26e2-97d4-42a2-8ede-022f6bb6900d admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:48.121862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb26e2-97d4-42a2-8ede-022f6bb6900d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:48.123818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb26e2-97d4-42a2-8ede-022f6bb6900d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:48.233855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1176/2370] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:48 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:48.238023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350cb68-86f5-494a-99bc-95d357418d07 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:48.253594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350cb68-86f5-494a-99bc-95d357418d07 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:48.260790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:48.358050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf0c15-5030-42c8-b38a-c231095f75d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:48.358896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf0c15-5030-42c8-b38a-c231095f75d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:48.359179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf0c15-5030-42c8-b38a-c231095f75d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:48.359456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf0c15-5030-42c8-b38a-c231095f75d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:48.360091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf0c15-5030-42c8-b38a-c231095f75d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:48.362048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf0c15-5030-42c8-b38a-c231095f75d2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:48.488440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1195/2371] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:48 2022] POST /identity/v3/users => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:48.492449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb26e2-97d4-42a2-8ede-022f6bb6900d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:48.506116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb26e2-97d4-42a2-8ede-022f6bb6900d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:48.511515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f8ac4-eab2-4e7d-bf85-33b291874ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:48.512918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f8ac4-eab2-4e7d-bf85-33b291874ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:48.513420 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f8ac4-eab2-4e7d-bf85-33b291874ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:48.513947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f8ac4-eab2-4e7d-bf85-33b291874ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:48.514835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f8ac4-eab2-4e7d-bf85-33b291874ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:48.516971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f8ac4-eab2-4e7d-bf85-33b291874ed1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:48.530760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1177/2372] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:48.535273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf0c15-5030-42c8-b38a-c231095f75d2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:48.540111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287c2d2-778f-465f-b860-f57107db5716 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:48.548742 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287c2d2-778f-465f-b860-f57107db5716 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:48.549554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287c2d2-778f-465f-b860-f57107db5716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:48.554191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf0c15-5030-42c8-b38a-c231095f75d2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:48.559169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61b161-889f-4857-8831-6b4a8a13b5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:48.560259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b161-889f-4857-8831-6b4a8a13b5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:48.560861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b161-889f-4857-8831-6b4a8a13b5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:48.561327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b161-889f-4857-8831-6b4a8a13b5d7 admin admin] PATH_INFO: `/v3/projects/958f75cfbe2c41349fa0e91fcd80e2f9/users/abc8b9fe160546faada4e40eed48047d/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:48.585221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e287c2d2-778f-465f-b860-f57107db5716 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:48.585221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e287c2d2-778f-465f-b860-f57107db5716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:48.590900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61b161-889f-4857-8831-6b4a8a13b5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=958f75cfbe2c41349fa0e91fcd80e2f9, user_id=abc8b9fe160546faada4e40eed48047d, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:48.593800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61b161-889f-4857-8831-6b4a8a13b5d7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:48.662327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1178/2373] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:51:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:51:48.667511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287c2d2-778f-465f-b860-f57107db5716 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:48.669015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e287c2d2-778f-465f-b860-f57107db5716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:48.684600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287c2d2-778f-465f-b860-f57107db5716 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:48.689384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:48.690072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:48.690354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:48.690629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] PATH_INFO: `/v3/projects/10e2169e3405486faf19ab6b70c74598` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:48.691945 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e2169e3405486faf19ab6b70c74598)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:48.694922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:48.757221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1196/2374] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:48 2022] PUT /identity/v3/projects/958f75cfbe2c41349fa0e91fcd80e2f9/users/abc8b9fe160546faada4e40eed48047d/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:48.757221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61b161-889f-4857-8831-6b4a8a13b5d7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:48.757221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2b61b161-889f-4857-8831-6b4a8a13b5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:48.867820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61b161-889f-4857-8831-6b4a8a13b5d7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:48.903046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e2169e3405486faf19ab6b70c74598', 'request_id': 'req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:48.954100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98408e75-4a31-47c0-a483-5b4982a5eaf4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:48.954909 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98408e75-4a31-47c0-a483-5b4982a5eaf4 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:48.955282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98408e75-4a31-47c0-a483-5b4982a5eaf4 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:48.955647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98408e75-4a31-47c0-a483-5b4982a5eaf4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:49.010692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e2169e3405486faf19ab6b70c74598', 'request_id': 'req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:49.016749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98408e75-4a31-47c0-a483-5b4982a5eaf4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:49.018359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98408e75-4a31-47c0-a483-5b4982a5eaf4 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:49.024780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e2169e3405486faf19ab6b70c74598', 'request_id': 'req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:49.027809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1197/2375] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:51:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:51:49.042593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] The token cache is being invalidate because project 10e2169e3405486faf19ab6b70c74598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:49.042727 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98408e75-4a31-47c0-a483-5b4982a5eaf4 service swift] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:49.042837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:49.042837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:49.043694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1179/2376] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:48 2022] DELETE /identity/v3/projects/10e2169e3405486faf19ab6b70c74598 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:49.053355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:49.054834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:49.135214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98408e75-4a31-47c0-a483-5b4982a5eaf4 service swift] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:49.140105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a472de-d3ba-4194-8a9d-d26b9e7fc4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:49.140881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a472de-d3ba-4194-8a9d-d26b9e7fc4cb admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:49.141171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a472de-d3ba-4194-8a9d-d26b9e7fc4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:49.141445 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a472de-d3ba-4194-8a9d-d26b9e7fc4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:49.142130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a472de-d3ba-4194-8a9d-d26b9e7fc4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:49.145442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a472de-d3ba-4194-8a9d-d26b9e7fc4cb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:49.158963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1198/2377] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:49.159375 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1cdfa-2b49-40ad-aeaf-73a387c1875d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:49.164007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a472de-d3ba-4194-8a9d-d26b9e7fc4cb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:49.164499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:49.165167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:49.165450 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:49.165816 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] PATH_INFO: `/v3/users/e687269edc5649098f4124444ac52fdf` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:49.167019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e687269edc5649098f4124444ac52fdf)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:49.167222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-50a472de-d3ba-4194-8a9d-d26b9e7fc4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:49.169123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:49.184824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a472de-d3ba-4194-8a9d-d26b9e7fc4cb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:49.189949 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79caf9d-c460-4eee-9e2d-d0735fa626e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:49.190621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79caf9d-c460-4eee-9e2d-d0735fa626e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:49.190905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79caf9d-c460-4eee-9e2d-d0735fa626e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:49.191182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79caf9d-c460-4eee-9e2d-d0735fa626e0 admin admin] PATH_INFO: `/v3/projects/958f75cfbe2c41349fa0e91fcd80e2f9/users/abc8b9fe160546faada4e40eed48047d/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:49.194683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79caf9d-c460-4eee-9e2d-d0735fa626e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=958f75cfbe2c41349fa0e91fcd80e2f9, user_id=abc8b9fe160546faada4e40eed48047d, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:49.197425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79caf9d-c460-4eee-9e2d-d0735fa626e0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:49.218940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1199/2378] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:49 2022] PUT /identity/v3/projects/958f75cfbe2c41349fa0e91fcd80e2f9/users/abc8b9fe160546faada4e40eed48047d/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:51:49.447906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e687269edc5649098f4124444ac52fdf'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:49.527036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e687269edc5649098f4124444ac52fdf'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:49.527953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e687269edc5649098f4124444ac52fdf'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:49.543478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e687269edc5649098f4124444ac52fdf'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:49.568719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-54ccb4b2-d7f4-400e-9c8b-29617248ec4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e687269edc5649098f4124444ac52fdf'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:49.582442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1180/2379] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:49 2022] DELETE /identity/v3/users/e687269edc5649098f4124444ac52fdf => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:49.874812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79caf9d-c460-4eee-9e2d-d0735fa626e0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:49.877483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a79caf9d-c460-4eee-9e2d-d0735fa626e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:50.031122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79caf9d-c460-4eee-9e2d-d0735fa626e0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:50.207535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:50.209169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:50.211289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:50.211289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] PATH_INFO: `/v3/users/693d8e1548e348c0aefe8ef7feb1f70c` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:50.212269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=693d8e1548e348c0aefe8ef7feb1f70c)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:50.221059 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:50.508292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '693d8e1548e348c0aefe8ef7feb1f70c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:50.541617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '693d8e1548e348c0aefe8ef7feb1f70c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:50.622471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '693d8e1548e348c0aefe8ef7feb1f70c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:50.623107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '693d8e1548e348c0aefe8ef7feb1f70c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:50.644513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '693d8e1548e348c0aefe8ef7feb1f70c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:50.659414 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1200/2380] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:49 2022] DELETE /identity/v3/users/693d8e1548e348c0aefe8ef7feb1f70c => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:50.661090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4eda5-91fd-461c-bf90-ba1041a78ef1 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:50.662489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4eda5-91fd-461c-bf90-ba1041a78ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:50.663028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4eda5-91fd-461c-bf90-ba1041a78ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:50.710613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-13e4eda5-91fd-461c-bf90-ba1041a78ef1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:50.712306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-13e4eda5-91fd-461c-bf90-ba1041a78ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:50.823864 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:50.954829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1181/2381] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:51.014633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:51.014633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:51.222321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbf40a-4138-4d2c-82ca-ed8dca7baa34 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:51.240648 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83573cae-e460-4d65-965c-b6efd0e156dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:51.241296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83573cae-e460-4d65-965c-b6efd0e156dc service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:51.241619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83573cae-e460-4d65-965c-b6efd0e156dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:51.241992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83573cae-e460-4d65-965c-b6efd0e156dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:51.291015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83573cae-e460-4d65-965c-b6efd0e156dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:51.298903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83573cae-e460-4d65-965c-b6efd0e156dc service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:51.308150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:51.346102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1201/2382] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:51.522116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4eda5-91fd-461c-bf90-ba1041a78ef1 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:51.524960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-13e4eda5-91fd-461c-bf90-ba1041a78ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:51.561038 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4eda5-91fd-461c-bf90-ba1041a78ef1 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:51.572056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:51.572056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:51.572056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:51.572056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] PATH_INFO: `/v3/projects/0287c5d55f564217a41ea6ff52d32d95` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:51.573074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0287c5d55f564217a41ea6ff52d32d95)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:51.593844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:51.846226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0287c5d55f564217a41ea6ff52d32d95', 'request_id': 'req-a9316307-602f-4347-80ea-7855b14db31f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:52.024324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0287c5d55f564217a41ea6ff52d32d95', 'request_id': 'req-a9316307-602f-4347-80ea-7855b14db31f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:52.060975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0287c5d55f564217a41ea6ff52d32d95', 'request_id': 'req-a9316307-602f-4347-80ea-7855b14db31f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:52.274350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] The token cache is being invalidate because project 0287c5d55f564217a41ea6ff52d32d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:52.274350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:52.274350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:52.274350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1182/2383] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:51 2022] DELETE /identity/v3/projects/0287c5d55f564217a41ea6ff52d32d95 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:52.276278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83573cae-e460-4d65-965c-b6efd0e156dc service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:52.289705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83573cae-e460-4d65-965c-b6efd0e156dc service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:52.382712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:52.382712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:52.382712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:52.382712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] PATH_INFO: `/v3/users/5a57c2ceeb5d40e3934647f2075a27c3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:52.384612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a57c2ceeb5d40e3934647f2075a27c3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:52.384612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:52.597612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:52.599553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:52.623968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9316307-602f-4347-80ea-7855b14db31f admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:52.639245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:52.640174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:52.640832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:52.641332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] PATH_INFO: `/v3/projects/d27f28ed6ca843178174ceeaa12be4bc` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:52.643136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27f28ed6ca843178174ceeaa12be4bc)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:52.646315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:52.860858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a57c2ceeb5d40e3934647f2075a27c3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:52.865279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27f28ed6ca843178174ceeaa12be4bc', 'request_id': 'req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:52.883467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a57c2ceeb5d40e3934647f2075a27c3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:52.961877 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27f28ed6ca843178174ceeaa12be4bc', 'request_id': 'req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:52.970246 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27f28ed6ca843178174ceeaa12be4bc', 'request_id': 'req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:53.002496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] The token cache is being invalidate because project d27f28ed6ca843178174ceeaa12be4bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:53.003197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:53.007858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-516b94ad-e9b1-4e7e-af8d-9cb256dc0d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:53.011896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1183/2384] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:52 2022] DELETE /identity/v3/projects/d27f28ed6ca843178174ceeaa12be4bc => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:51:53.017398 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a214e1-9278-4771-9264-eec1326c7cef None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:53.017855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a214e1-9278-4771-9264-eec1326c7cef None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:53.018236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a214e1-9278-4771-9264-eec1326c7cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:53.071217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a57c2ceeb5d40e3934647f2075a27c3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:53.071869 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a57c2ceeb5d40e3934647f2075a27c3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:53.072745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-27a214e1-9278-4771-9264-eec1326c7cef None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:53.081164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-27a214e1-9278-4771-9264-eec1326c7cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:53.087248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cf41b2a1-b645-4823-8d38-7b1aebb71d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a57c2ceeb5d40e3934647f2075a27c3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:53.114884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1202/2385] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:51:52 2022] DELETE /identity/v3/users/5a57c2ceeb5d40e3934647f2075a27c3 => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:53.242765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1184/2386] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:53.245360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d287d89-d7ff-4163-b164-bb537ed23177 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:53.245485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d287d89-d7ff-4163-b164-bb537ed23177 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:53.245668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d287d89-d7ff-4163-b164-bb537ed23177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:53.296272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:53.300368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8d287d89-d7ff-4163-b164-bb537ed23177 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:53.302665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8d287d89-d7ff-4163-b164-bb537ed23177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:53.579319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1203/2387] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:53.587846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a214e1-9278-4771-9264-eec1326c7cef None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:53.604171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a214e1-9278-4771-9264-eec1326c7cef None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:53.614229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c328-0a45-4e44-829c-f344a011f282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:53.616405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c328-0a45-4e44-829c-f344a011f282 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:53.616405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c328-0a45-4e44-829c-f344a011f282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:53.616405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c328-0a45-4e44-829c-f344a011f282 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:53.616405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4c328-0a45-4e44-829c-f344a011f282 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:51:53.617317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4c328-0a45-4e44-829c-f344a011f282 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:53.621824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4c328-0a45-4e44-829c-f344a011f282 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:53.639396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:53.682510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1185/2388] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:51:53.683774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d287d89-d7ff-4163-b164-bb537ed23177 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:53.697603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d287d89-d7ff-4163-b164-bb537ed23177 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:53.710820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cebda1-0826-45de-b3f1-9ec53d9498fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:53.711453 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cebda1-0826-45de-b3f1-9ec53d9498fd admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:53.711735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cebda1-0826-45de-b3f1-9ec53d9498fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:53.712031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cebda1-0826-45de-b3f1-9ec53d9498fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:53.712684 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cebda1-0826-45de-b3f1-9ec53d9498fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:53.714913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cebda1-0826-45de-b3f1-9ec53d9498fd admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:53.851492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1204/2389] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:53 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:51:53.854960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c328-0a45-4e44-829c-f344a011f282 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:53.884455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c328-0a45-4e44-829c-f344a011f282 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:54.079588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0033a-d3dc-48a4-94d7-3d6e69ffbd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:54.080216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0033a-d3dc-48a4-94d7-3d6e69ffbd36 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:54.080817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0033a-d3dc-48a4-94d7-3d6e69ffbd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:54.081488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0033a-d3dc-48a4-94d7-3d6e69ffbd36 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:54.082659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0033a-d3dc-48a4-94d7-3d6e69ffbd36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:54.086112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0033a-d3dc-48a4-94d7-3d6e69ffbd36 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:54.195559 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cebda1-0826-45de-b3f1-9ec53d9498fd admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:54.197164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-98cebda1-0826-45de-b3f1-9ec53d9498fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:54.202307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1186/2390] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:53 2022] POST /identity/v3/users => generated 287 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:54.210784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0033a-d3dc-48a4-94d7-3d6e69ffbd36 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:54.234179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cebda1-0826-45de-b3f1-9ec53d9498fd admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:54.242830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0033a-d3dc-48a4-94d7-3d6e69ffbd36 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:54.260276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:54.261209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:54.264175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:54.264642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] PATH_INFO: `/v3/projects/360f6e55040a42deb6dde6d90416e4e5` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:54.272754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=360f6e55040a42deb6dde6d90416e4e5)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:54.274190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5b137-c011-4b38-be78-3b27296ea74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:54.276654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b137-c011-4b38-be78-3b27296ea74c admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:54.277161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b137-c011-4b38-be78-3b27296ea74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:54.277809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:54.278270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b137-c011-4b38-be78-3b27296ea74c admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:54.282307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5b137-c011-4b38-be78-3b27296ea74c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:54.288691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5b137-c011-4b38-be78-3b27296ea74c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:54.314309 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1187/2391] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:54.321482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5b137-c011-4b38-be78-3b27296ea74c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:54.340170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5b137-c011-4b38-be78-3b27296ea74c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:54.351251 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5987282-4546-4cbb-91cf-ada387fc201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:54.358322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5987282-4546-4cbb-91cf-ada387fc201e admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:54.358322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5987282-4546-4cbb-91cf-ada387fc201e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:54.358322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5987282-4546-4cbb-91cf-ada387fc201e admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:54.358322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5987282-4546-4cbb-91cf-ada387fc201e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:54.358322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5987282-4546-4cbb-91cf-ada387fc201e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:54.383401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1188/2392] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:54.384681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5987282-4546-4cbb-91cf-ada387fc201e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:54.410196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5987282-4546-4cbb-91cf-ada387fc201e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:54.418559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863967c7-95b8-4e8f-b1ac-afc883defff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:54.424131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863967c7-95b8-4e8f-b1ac-afc883defff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:54.424131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863967c7-95b8-4e8f-b1ac-afc883defff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:54.424131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863967c7-95b8-4e8f-b1ac-afc883defff4 admin admin] PATH_INFO: `/v3/projects/c265d90202534462b37c19391538b29a/users/4a26f3f51f9c4e4aa4b1d7ce8bc041d6/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:54.454305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863967c7-95b8-4e8f-b1ac-afc883defff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c265d90202534462b37c19391538b29a, user_id=4a26f3f51f9c4e4aa4b1d7ce8bc041d6, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:54.458516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863967c7-95b8-4e8f-b1ac-afc883defff4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:54.592593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1189/2393] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:54 2022] PUT /identity/v3/projects/c265d90202534462b37c19391538b29a/users/4a26f3f51f9c4e4aa4b1d7ce8bc041d6/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:54.594586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f5ea-c883-4022-be94-73f19f1ce836 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:54.594877 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f5ea-c883-4022-be94-73f19f1ce836 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:54.595203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f5ea-c883-4022-be94-73f19f1ce836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:54.595419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '360f6e55040a42deb6dde6d90416e4e5', 'request_id': 'req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:54.610156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '360f6e55040a42deb6dde6d90416e4e5', 'request_id': 'req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:54.626000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-db52f5ea-c883-4022-be94-73f19f1ce836 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:54.629364 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-db52f5ea-c883-4022-be94-73f19f1ce836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:54.716096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '360f6e55040a42deb6dde6d90416e4e5', 'request_id': 'req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:54.720679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:54.740695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] The token cache is being invalidate because project 360f6e55040a42deb6dde6d90416e4e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:51:54.743568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:54.743689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1190/2394] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:54.743819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:51:54.750331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1205/2395] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:51:54 2022] DELETE /identity/v3/projects/360f6e55040a42deb6dde6d90416e4e5 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:51:54.764737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574cbcc-bd94-4d50-beaf-faabfc5c122e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:54.769548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:54.769701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574cbcc-bd94-4d50-beaf-faabfc5c122e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:54.769701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574cbcc-bd94-4d50-beaf-faabfc5c122e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:54.769928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:54.822459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6574cbcc-bd94-4d50-beaf-faabfc5c122e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:54.825090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6574cbcc-bd94-4d50-beaf-faabfc5c122e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:54.836469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1191/2396] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:54.848390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fab0f-b26c-4291-81fe-aa9282da8a81 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:54.859349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fab0f-b26c-4291-81fe-aa9282da8a81 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:54.859897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fab0f-b26c-4291-81fe-aa9282da8a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:54.903696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-064fab0f-b26c-4291-81fe-aa9282da8a81 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:54.924750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-064fab0f-b26c-4291-81fe-aa9282da8a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:54.960935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1192/2397] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:54.962110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fab0f-b26c-4291-81fe-aa9282da8a81 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:54.964089 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb6d28-895e-4b68-9bab-48d51bfadd1c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:54.975184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeea748-17d4-4d8e-a3a5-f71b36385660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:54.978581 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeea748-17d4-4d8e-a3a5-f71b36385660 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:54.978862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeea748-17d4-4d8e-a3a5-f71b36385660 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:54.980632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeea748-17d4-4d8e-a3a5-f71b36385660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:54.986432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-faeea748-17d4-4d8e-a3a5-f71b36385660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:54.988967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fab0f-b26c-4291-81fe-aa9282da8a81 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:55.001954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4a65c-3fbc-4908-91fc-08ac320c0945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:55.009327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4a65c-3fbc-4908-91fc-08ac320c0945 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:55.009327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4a65c-3fbc-4908-91fc-08ac320c0945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:55.009327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4a65c-3fbc-4908-91fc-08ac320c0945 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:55.009327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4a65c-3fbc-4908-91fc-08ac320c0945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:51:55.009327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4a65c-3fbc-4908-91fc-08ac320c0945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:55.009327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4a65c-3fbc-4908-91fc-08ac320c0945 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:55.039504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeea748-17d4-4d8e-a3a5-f71b36385660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:55.046635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeea748-17d4-4d8e-a3a5-f71b36385660 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:55.049884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1193/2398] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:51:55.052375 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4a65c-3fbc-4908-91fc-08ac320c0945 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:55.087556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4a65c-3fbc-4908-91fc-08ac320c0945 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:55.090585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1206/2399] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:51:55.112519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fb0e7-63c1-40ad-9a8c-42d74c0cfeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:55.112519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fb0e7-63c1-40ad-9a8c-42d74c0cfeab admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:55.112519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fb0e7-63c1-40ad-9a8c-42d74c0cfeab admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:55.112519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fb0e7-63c1-40ad-9a8c-42d74c0cfeab admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:55.113304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fb0e7-63c1-40ad-9a8c-42d74c0cfeab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:55.113304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fb0e7-63c1-40ad-9a8c-42d74c0cfeab admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:55.895188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1194/2400] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:51:55 2022] POST /identity/v3/projects => generated 375 bytes in 844 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:55.902224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeea748-17d4-4d8e-a3a5-f71b36385660 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:55.915436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeea748-17d4-4d8e-a3a5-f71b36385660 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:56.060913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5ca0e-0dd2-488a-b5e9-419f8cb6ddec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:56.062568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5ca0e-0dd2-488a-b5e9-419f8cb6ddec admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:56.062985 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5ca0e-0dd2-488a-b5e9-419f8cb6ddec admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:56.063182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5ca0e-0dd2-488a-b5e9-419f8cb6ddec admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:56.068424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5ca0e-0dd2-488a-b5e9-419f8cb6ddec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:56.068424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5ca0e-0dd2-488a-b5e9-419f8cb6ddec admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:56.197717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1207/2401] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:51:55 2022] POST /identity/v3/users => generated 288 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:51:56.204733 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fb0e7-63c1-40ad-9a8c-42d74c0cfeab admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:56.220691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fb0e7-63c1-40ad-9a8c-42d74c0cfeab admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:56.235011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cd56-6953-4535-878b-e0b303a06ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:56.235749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cd56-6953-4535-878b-e0b303a06ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:56.236028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cd56-6953-4535-878b-e0b303a06ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:56.237259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cd56-6953-4535-878b-e0b303a06ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:56.238492 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813cd56-6953-4535-878b-e0b303a06ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:56.250231 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813cd56-6953-4535-878b-e0b303a06ed3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:56.275252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5ca0e-0dd2-488a-b5e9-419f8cb6ddec admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:56.275528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1195/2402] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:51:56.299225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5ca0e-0dd2-488a-b5e9-419f8cb6ddec admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:56.319355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:56.339656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29686af-3bfa-4d14-b762-33f3878bf37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:56.339656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29686af-3bfa-4d14-b762-33f3878bf37d admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:56.339656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29686af-3bfa-4d14-b762-33f3878bf37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:56.339656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29686af-3bfa-4d14-b762-33f3878bf37d admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:56.340169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29686af-3bfa-4d14-b762-33f3878bf37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:56.340169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29686af-3bfa-4d14-b762-33f3878bf37d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:56.388521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1208/2403] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:51:56.396596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cd56-6953-4535-878b-e0b303a06ed3 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:56.412249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cd56-6953-4535-878b-e0b303a06ed3 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:56.425747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ea392-11a9-44d8-a7ef-15634d21a344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:56.427471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ea392-11a9-44d8-a7ef-15634d21a344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:56.427925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ea392-11a9-44d8-a7ef-15634d21a344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:56.428373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ea392-11a9-44d8-a7ef-15634d21a344 admin admin] PATH_INFO: `/v3/projects/2c738e90367f4037bc9d102fb030ba9e/users/db33c8e52e784150be3361cc9f8456c1/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:56.468944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ea392-11a9-44d8-a7ef-15634d21a344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c738e90367f4037bc9d102fb030ba9e, user_id=db33c8e52e784150be3361cc9f8456c1, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:56.472523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ea392-11a9-44d8-a7ef-15634d21a344 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:56.558183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1196/2404] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:51:56 2022] PUT /identity/v3/projects/2c738e90367f4037bc9d102fb030ba9e/users/db33c8e52e784150be3361cc9f8456c1/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:51:56.571161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa572e3-1f27-4598-9fdd-5160ac2605f1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:56.571731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa572e3-1f27-4598-9fdd-5160ac2605f1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:56.572341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa572e3-1f27-4598-9fdd-5160ac2605f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:56.652466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5aa572e3-1f27-4598-9fdd-5160ac2605f1 None None] MFA Rules not processed for user `db33c8e52e784150be3361cc9f8456c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:56.658624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5aa572e3-1f27-4598-9fdd-5160ac2605f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:56.879063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1209/2405] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:51:56.893245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ea392-11a9-44d8-a7ef-15634d21a344 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:51:56.899028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e53ea392-11a9-44d8-a7ef-15634d21a344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:56.991211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:57.063670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ea392-11a9-44d8-a7ef-15634d21a344 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:51:57.083945 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67a63c-b612-45d2-a0ec-5924e2fae640 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab54ee52004746eaadb5c2970e59afc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:51:57.085054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67a63c-b612-45d2-a0ec-5924e2fae640 service glance] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:57.085941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67a63c-b612-45d2-a0ec-5924e2fae640 service glance] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:57.088408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67a63c-b612-45d2-a0ec-5924e2fae640 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:57.109906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67a63c-b612-45d2-a0ec-5924e2fae640 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:51:57.139613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67a63c-b612-45d2-a0ec-5924e2fae640 service glance] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:51:57.167835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1197/2406] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:51:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:51:57.868544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:58.978895 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:51:59.150239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603b710-5bec-4820-a8d7-b8b06b66e4af None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:51:59.152207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603b710-5bec-4820-a8d7-b8b06b66e4af None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:51:59.152207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603b710-5bec-4820-a8d7-b8b06b66e4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:51:59.187826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b603b710-5bec-4820-a8d7-b8b06b66e4af None None] MFA Rules not processed for user `89faad7b1c26412cac5c2c624fa3f0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:51:59.191844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b603b710-5bec-4820-a8d7-b8b06b66e4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:51:59.358455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1210/2407] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:51:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:03.700345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d684e2b-0ca7-4e2c-b161-e6af229d4060 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:03.702716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d684e2b-0ca7-4e2c-b161-e6af229d4060 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:03.702716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d684e2b-0ca7-4e2c-b161-e6af229d4060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:03.770503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4d684e2b-0ca7-4e2c-b161-e6af229d4060 None None] MFA Rules not processed for user `abc8b9fe160546faada4e40eed48047d`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:03.771924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4d684e2b-0ca7-4e2c-b161-e6af229d4060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:04.004459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1198/2408] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:04.016653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603b710-5bec-4820-a8d7-b8b06b66e4af None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:04.019717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b603b710-5bec-4820-a8d7-b8b06b66e4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:04.060395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603b710-5bec-4820-a8d7-b8b06b66e4af None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:04.071466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025f686-ee83-4739-ac83-36dba5d5cf7e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab54ee52004746eaadb5c2970e59afc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:04.074659 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025f686-ee83-4739-ac83-36dba5d5cf7e service glance] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:04.075607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025f686-ee83-4739-ac83-36dba5d5cf7e service glance] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:04.076061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025f686-ee83-4739-ac83-36dba5d5cf7e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:04.080710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7025f686-ee83-4739-ac83-36dba5d5cf7e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:04.276003 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025f686-ee83-4739-ac83-36dba5d5cf7e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:04.278101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025f686-ee83-4739-ac83-36dba5d5cf7e service glance] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:04.307043 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1211/2409] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:04.628259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d684e2b-0ca7-4e2c-b161-e6af229d4060 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:04.630409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4d684e2b-0ca7-4e2c-b161-e6af229d4060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:04.694497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a3f44-ace4-4bdb-954c-24ba418a7e4b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:04.695295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a3f44-ace4-4bdb-954c-24ba418a7e4b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:04.695832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a3f44-ace4-4bdb-954c-24ba418a7e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:04.730304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:04.735337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9a4a3f44-ace4-4bdb-954c-24ba418a7e4b None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:04.737383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9a4a3f44-ace4-4bdb-954c-24ba418a7e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:04.785226 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d684e2b-0ca7-4e2c-b161-e6af229d4060 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:04.794264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a554c5-cb23-4f18-a450-0768ace815e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ec5c9c71d14762a4869b7d66e87ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:04.795200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a554c5-cb23-4f18-a450-0768ace815e6 service placement] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:04.795631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a554c5-cb23-4f18-a450-0768ace815e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:04.796051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a554c5-cb23-4f18-a450-0768ace815e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:04.804638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d7a554c5-cb23-4f18-a450-0768ace815e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:04.820362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a554c5-cb23-4f18-a450-0768ace815e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:04.822148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a554c5-cb23-4f18-a450-0768ace815e6 service placement] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:04.844388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1199/2410] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:04.873880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1212/2411] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:52:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:52:04.898432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a554c5-cb23-4f18-a450-0768ace815e6 service placement] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:04.899411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d7a554c5-cb23-4f18-a450-0768ace815e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:05.076014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a554c5-cb23-4f18-a450-0768ace815e6 service placement] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:05.081328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e196f-8cfa-45ab-b643-41b0b3f6a008 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:05.081935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e196f-8cfa-45ab-b643-41b0b3f6a008 service swift] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:05.082218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e196f-8cfa-45ab-b643-41b0b3f6a008 service swift] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:05.086298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e196f-8cfa-45ab-b643-41b0b3f6a008 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:05.095840 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e196f-8cfa-45ab-b643-41b0b3f6a008 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:05.097454 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e196f-8cfa-45ab-b643-41b0b3f6a008 service swift] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:05.112189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1200/2412] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:52:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:52:06.407736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c35d9-c4e9-42b0-b84a-67ca5c2f6259 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:06.409811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c35d9-c4e9-42b0-b84a-67ca5c2f6259 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:06.410260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c35d9-c4e9-42b0-b84a-67ca5c2f6259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:06.478686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-af2c35d9-c4e9-42b0-b84a-67ca5c2f6259 None None] MFA Rules not processed for user `4a26f3f51f9c4e4aa4b1d7ce8bc041d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:06.480613 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-af2c35d9-c4e9-42b0-b84a-67ca5c2f6259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:06.570229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:06.684859 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1213/2413] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:06.687669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e196f-8cfa-45ab-b643-41b0b3f6a008 service swift] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:06.690343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-de0e196f-8cfa-45ab-b643-41b0b3f6a008 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:06.871675 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e196f-8cfa-45ab-b643-41b0b3f6a008 service swift] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:06.933601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ddc41-edf7-49af-ba37-f519fe0ec565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:06.934339 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ddc41-edf7-49af-ba37-f519fe0ec565 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:06.934706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ddc41-edf7-49af-ba37-f519fe0ec565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:06.935041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ddc41-edf7-49af-ba37-f519fe0ec565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:06.935800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ddc41-edf7-49af-ba37-f519fe0ec565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:06.939491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ddc41-edf7-49af-ba37-f519fe0ec565 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:07.716789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1201/2414] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:52:06 2022] POST /identity/v3/projects => generated 371 bytes in 1028 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:52:07.717976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c35d9-c4e9-42b0-b84a-67ca5c2f6259 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:07.735222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c35d9-c4e9-42b0-b84a-67ca5c2f6259 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:07.909454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad712b9-63c1-4240-b15b-74806d47271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:07.910463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad712b9-63c1-4240-b15b-74806d47271b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:07.911374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad712b9-63c1-4240-b15b-74806d47271b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:07.911775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad712b9-63c1-4240-b15b-74806d47271b admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:07.913203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad712b9-63c1-4240-b15b-74806d47271b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:07.916537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad712b9-63c1-4240-b15b-74806d47271b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:08.039586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1214/2415] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:52:07 2022] POST /identity/v3/users => generated 292 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:52:08.046389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ddc41-edf7-49af-ba37-f519fe0ec565 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:08.064285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ddc41-edf7-49af-ba37-f519fe0ec565 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:08.070965 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf451f-da0b-45f9-a3ac-39ea9f003be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:08.071647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf451f-da0b-45f9-a3ac-39ea9f003be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:08.071944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf451f-da0b-45f9-a3ac-39ea9f003be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:08.072217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf451f-da0b-45f9-a3ac-39ea9f003be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:08.072975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf451f-da0b-45f9-a3ac-39ea9f003be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:08.074856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf451f-da0b-45f9-a3ac-39ea9f003be2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:08.095499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1202/2416] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:52:08.099100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad712b9-63c1-4240-b15b-74806d47271b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:08.110853 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad712b9-63c1-4240-b15b-74806d47271b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:08.121877 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afae1f3-7f6f-4306-9f70-ec61debe1b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:08.122875 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afae1f3-7f6f-4306-9f70-ec61debe1b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:08.123564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afae1f3-7f6f-4306-9f70-ec61debe1b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:08.124043 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afae1f3-7f6f-4306-9f70-ec61debe1b3c admin admin] PATH_INFO: `/v3/projects/6d045ef7133244a38cb14ad861dcc8d2/users/3a91cf98350d446daf3a5ac6c462d5be/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:08.142740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afae1f3-7f6f-4306-9f70-ec61debe1b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d045ef7133244a38cb14ad861dcc8d2, user_id=3a91cf98350d446daf3a5ac6c462d5be, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:08.147980 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afae1f3-7f6f-4306-9f70-ec61debe1b3c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:08.285272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1215/2417] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:52:08 2022] PUT /identity/v3/projects/6d045ef7133244a38cb14ad861dcc8d2/users/3a91cf98350d446daf3a5ac6c462d5be/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:52:08.289782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf451f-da0b-45f9-a3ac-39ea9f003be2 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:08.324716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf451f-da0b-45f9-a3ac-39ea9f003be2 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:08.397119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:08.533296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b4484-353a-4eae-be02-ad3a23d33aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:08.539361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b4484-353a-4eae-be02-ad3a23d33aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:08.542473 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b4484-353a-4eae-be02-ad3a23d33aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:08.543391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b4484-353a-4eae-be02-ad3a23d33aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:08.546516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b4484-353a-4eae-be02-ad3a23d33aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:08.555831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b4484-353a-4eae-be02-ad3a23d33aa6 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:08.628715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1203/2418] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:52:08.634066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afae1f3-7f6f-4306-9f70-ec61debe1b3c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:08.664747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afae1f3-7f6f-4306-9f70-ec61debe1b3c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:08.675280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c8117-d4a0-4070-8cee-0a45d62f5ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:08.677823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c8117-d4a0-4070-8cee-0a45d62f5ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:08.680634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c8117-d4a0-4070-8cee-0a45d62f5ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:08.680634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c8117-d4a0-4070-8cee-0a45d62f5ceb admin admin] PATH_INFO: `/v3/projects/6d045ef7133244a38cb14ad861dcc8d2/users/3a91cf98350d446daf3a5ac6c462d5be/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:08.683177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c8117-d4a0-4070-8cee-0a45d62f5ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d045ef7133244a38cb14ad861dcc8d2, user_id=3a91cf98350d446daf3a5ac6c462d5be, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:08.689170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c8117-d4a0-4070-8cee-0a45d62f5ceb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:08.726048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1216/2419] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:52:08 2022] PUT /identity/v3/projects/6d045ef7133244a38cb14ad861dcc8d2/users/3a91cf98350d446daf3a5ac6c462d5be/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:17.579576 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460f480-9d85-4460-94dc-898b12c38c32 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:17.591444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460f480-9d85-4460-94dc-898b12c38c32 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:17.591444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460f480-9d85-4460-94dc-898b12c38c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:17.612230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-7460f480-9d85-4460-94dc-898b12c38c32 None None] MFA Rules not processed for user `3a91cf98350d446daf3a5ac6c462d5be`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:17.613467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7460f480-9d85-4460-94dc-898b12c38c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:17.715072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1204/2420] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:17.729055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c8117-d4a0-4070-8cee-0a45d62f5ceb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:17.730885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f35c8117-d4a0-4070-8cee-0a45d62f5ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:17.830973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c8117-d4a0-4070-8cee-0a45d62f5ceb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:17.841901 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c311214-7761-4287-ba01-fc613f57d9c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:17.842545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c311214-7761-4287-ba01-fc613f57d9c0 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:17.845606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c311214-7761-4287-ba01-fc613f57d9c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:17.845606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c311214-7761-4287-ba01-fc613f57d9c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:17.872651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:17.927732 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c311214-7761-4287-ba01-fc613f57d9c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:17.936534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c311214-7761-4287-ba01-fc613f57d9c0 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:17.950002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1217/2421] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:18.931510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b28b92-9655-4930-b3a8-33b78e58d534 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:18.938130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b28b92-9655-4930-b3a8-33b78e58d534 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:18.938130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b28b92-9655-4930-b3a8-33b78e58d534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:18.956610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-c7b28b92-9655-4930-b3a8-33b78e58d534 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:18.958876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c7b28b92-9655-4930-b3a8-33b78e58d534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:19.033567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1205/2422] 10.222.0.42 () {58 vars in 967 bytes} [Sat Sep 17 03:52:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:19.057852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c311214-7761-4287-ba01-fc613f57d9c0 service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:19.199313 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c311214-7761-4287-ba01-fc613f57d9c0 service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:19.206226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cb603-58c8-435d-a16a-35a2093409f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:19.206970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cb603-58c8-435d-a16a-35a2093409f9 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:19.207350 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cb603-58c8-435d-a16a-35a2093409f9 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:19.207749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cb603-58c8-435d-a16a-35a2093409f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:19.218885 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cb603-58c8-435d-a16a-35a2093409f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:19.220679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cb603-58c8-435d-a16a-35a2093409f9 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:19.233676 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1218/2423] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:52:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:52:19.609250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b28b92-9655-4930-b3a8-33b78e58d534 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:19.706071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:19.714881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b28b92-9655-4930-b3a8-33b78e58d534 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:19.721187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:19.721938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:19.722287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:19.722654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] PATH_INFO: `/v3/users/db33c8e52e784150be3361cc9f8456c1` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:19.724277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db33c8e52e784150be3361cc9f8456c1)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:19.726189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:19.957102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db33c8e52e784150be3361cc9f8456c1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:20.028881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db33c8e52e784150be3361cc9f8456c1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:20.030523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db33c8e52e784150be3361cc9f8456c1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:20.039079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db33c8e52e784150be3361cc9f8456c1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:20.057013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db33c8e52e784150be3361cc9f8456c1'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:20.065817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1206/2424] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:52:19 2022] DELETE /identity/v3/users/db33c8e52e784150be3361cc9f8456c1 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:20.071209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768cd85-668d-498e-ae38-5961e22a147c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:20.071713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768cd85-668d-498e-ae38-5961e22a147c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:20.071801 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768cd85-668d-498e-ae38-5961e22a147c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:20.093809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e768cd85-668d-498e-ae38-5961e22a147c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:20.094955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e768cd85-668d-498e-ae38-5961e22a147c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:20.218892 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1219/2425] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:20.226629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:20.230689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:20.311054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3586a-763d-4385-b527-05f4f7349770 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:20.321796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce011fd1-97c1-4ba4-b9c4-8fb7c490533a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:20.332337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce011fd1-97c1-4ba4-b9c4-8fb7c490533a service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:20.332337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce011fd1-97c1-4ba4-b9c4-8fb7c490533a service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:20.332337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce011fd1-97c1-4ba4-b9c4-8fb7c490533a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:20.338241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce011fd1-97c1-4ba4-b9c4-8fb7c490533a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:20.340359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce011fd1-97c1-4ba4-b9c4-8fb7c490533a service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:20.353411 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1207/2426] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:21.416194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768cd85-668d-498e-ae38-5961e22a147c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:21.417631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e768cd85-668d-498e-ae38-5961e22a147c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:21.444150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768cd85-668d-498e-ae38-5961e22a147c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:21.462612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:21.466812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:21.467111 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:21.467400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] PATH_INFO: `/v3/projects/2c738e90367f4037bc9d102fb030ba9e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:21.471485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c738e90367f4037bc9d102fb030ba9e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:21.474955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:21.700157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:21.714820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c738e90367f4037bc9d102fb030ba9e', 'request_id': 'req-051d1214-2db0-430f-b8b2-9ebe18c4225a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:21.728650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c738e90367f4037bc9d102fb030ba9e', 'request_id': 'req-051d1214-2db0-430f-b8b2-9ebe18c4225a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:21.785845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751d9fe-4041-4acb-b113-827fdb9eaeb7 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:21.786736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751d9fe-4041-4acb-b113-827fdb9eaeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:21.787162 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751d9fe-4041-4acb-b113-827fdb9eaeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:21.849975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6751d9fe-4041-4acb-b113-827fdb9eaeb7 None None] MFA Rules not processed for user `4a26f3f51f9c4e4aa4b1d7ce8bc041d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:21.851206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6751d9fe-4041-4acb-b113-827fdb9eaeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:21.873524 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c738e90367f4037bc9d102fb030ba9e', 'request_id': 'req-051d1214-2db0-430f-b8b2-9ebe18c4225a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:21.903107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] The token cache is being invalidate because project 2c738e90367f4037bc9d102fb030ba9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:52:21.903398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:21.911639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-051d1214-2db0-430f-b8b2-9ebe18c4225a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:21.913286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1220/2427] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:52:21 2022] DELETE /identity/v3/projects/2c738e90367f4037bc9d102fb030ba9e => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:21.918033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe6c6-778b-4481-84d5-7a819e3cf2d4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:21.918512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe6c6-778b-4481-84d5-7a819e3cf2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:21.918943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe6c6-778b-4481-84d5-7a819e3cf2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:21.954810 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-791fe6c6-778b-4481-84d5-7a819e3cf2d4 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:21.958621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-791fe6c6-778b-4481-84d5-7a819e3cf2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:22.011322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1208/2428] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:52:22.103316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1221/2429] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:52:22.105761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df2d09-93ef-4d2f-aa8f-739768f2f486 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:22.105761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df2d09-93ef-4d2f-aa8f-739768f2f486 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:22.105998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df2d09-93ef-4d2f-aa8f-739768f2f486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:22.157280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-97df2d09-93ef-4d2f-aa8f-739768f2f486 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:22.163410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-97df2d09-93ef-4d2f-aa8f-739768f2f486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:22.194180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1209/2430] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:22.195859 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fe6c6-778b-4481-84d5-7a819e3cf2d4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:22.209314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fe6c6-778b-4481-84d5-7a819e3cf2d4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:22.231005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c02ac5-e234-428b-b010-949c4b67f324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:22.231849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c02ac5-e234-428b-b010-949c4b67f324 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:22.232143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c02ac5-e234-428b-b010-949c4b67f324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:22.232439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c02ac5-e234-428b-b010-949c4b67f324 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:22.233116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c02ac5-e234-428b-b010-949c4b67f324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:52:22.233422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c02ac5-e234-428b-b010-949c4b67f324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:22.235704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c02ac5-e234-428b-b010-949c4b67f324 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:22.271010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1222/2431] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:52:22.276754 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df2d09-93ef-4d2f-aa8f-739768f2f486 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:22.291418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df2d09-93ef-4d2f-aa8f-739768f2f486 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:22.305881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9555de-5cb3-4ac7-91cc-e2e792b7670a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:22.333853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9555de-5cb3-4ac7-91cc-e2e792b7670a admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:22.333853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9555de-5cb3-4ac7-91cc-e2e792b7670a admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:22.333853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9555de-5cb3-4ac7-91cc-e2e792b7670a admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:22.333853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9555de-5cb3-4ac7-91cc-e2e792b7670a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:22.333853 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9555de-5cb3-4ac7-91cc-e2e792b7670a admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:22.362527 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:22.641056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1210/2432] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:52:22 2022] POST /identity/v3/projects => generated 383 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:52:22.645587 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c02ac5-e234-428b-b010-949c4b67f324 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:22.655561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c02ac5-e234-428b-b010-949c4b67f324 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:22.717995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecda4c-f9d2-442c-aebe-64049205184b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:22.724208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecda4c-f9d2-442c-aebe-64049205184b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:22.724208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecda4c-f9d2-442c-aebe-64049205184b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:22.724208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecda4c-f9d2-442c-aebe-64049205184b admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:22.724208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecda4c-f9d2-442c-aebe-64049205184b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:22.724208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecda4c-f9d2-442c-aebe-64049205184b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:22.974183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1223/2433] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:52:22 2022] POST /identity/v3/users => generated 292 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:52:22.982150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9555de-5cb3-4ac7-91cc-e2e792b7670a admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:22.993691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9555de-5cb3-4ac7-91cc-e2e792b7670a admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:22.998385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c624733-0e57-45b2-9ff9-bd242ae3765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:22.998993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c624733-0e57-45b2-9ff9-bd242ae3765e admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:22.999442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c624733-0e57-45b2-9ff9-bd242ae3765e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:22.999726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c624733-0e57-45b2-9ff9-bd242ae3765e admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:23.000354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c624733-0e57-45b2-9ff9-bd242ae3765e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:23.002147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c624733-0e57-45b2-9ff9-bd242ae3765e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:23.015207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1211/2434] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:52:23.022204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecda4c-f9d2-442c-aebe-64049205184b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:23.032678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecda4c-f9d2-442c-aebe-64049205184b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:23.040779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94d66d-4289-4a28-b00b-7eaa9bea65ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:23.041374 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94d66d-4289-4a28-b00b-7eaa9bea65ad admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:23.041651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94d66d-4289-4a28-b00b-7eaa9bea65ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:23.041968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94d66d-4289-4a28-b00b-7eaa9bea65ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:23.042588 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94d66d-4289-4a28-b00b-7eaa9bea65ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:23.044167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94d66d-4289-4a28-b00b-7eaa9bea65ad admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:23.056188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1224/2435] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:52:23.060400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c624733-0e57-45b2-9ff9-bd242ae3765e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:23.070421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c624733-0e57-45b2-9ff9-bd242ae3765e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:23.075177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542f140-8ce3-4689-8777-f301de81cc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:23.075821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542f140-8ce3-4689-8777-f301de81cc66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:23.076097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542f140-8ce3-4689-8777-f301de81cc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:23.076368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542f140-8ce3-4689-8777-f301de81cc66 admin admin] PATH_INFO: `/v3/projects/28fb779c268540e9b0d71bbffd80c32f/users/28476ef1072a4f998abfa3bcf0df3dd0/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:23.093109 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542f140-8ce3-4689-8777-f301de81cc66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28fb779c268540e9b0d71bbffd80c32f, user_id=28476ef1072a4f998abfa3bcf0df3dd0, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:23.095786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542f140-8ce3-4689-8777-f301de81cc66 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:23.223186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1212/2436] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:52:23 2022] PUT /identity/v3/projects/28fb779c268540e9b0d71bbffd80c32f/users/28476ef1072a4f998abfa3bcf0df3dd0/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:23.240202 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1fc5-2b12-4aeb-8240-0abb2fbded9a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:23.240416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1fc5-2b12-4aeb-8240-0abb2fbded9a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:23.240685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1fc5-2b12-4aeb-8240-0abb2fbded9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:23.291773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-01ab1fc5-2b12-4aeb-8240-0abb2fbded9a None None] MFA Rules not processed for user `28476ef1072a4f998abfa3bcf0df3dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:23.293445 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-01ab1fc5-2b12-4aeb-8240-0abb2fbded9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:23.356522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:23.508800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1225/2437] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:23.514442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542f140-8ce3-4689-8777-f301de81cc66 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:23.680132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542f140-8ce3-4689-8777-f301de81cc66 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:23.694707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad1f86-5d49-4bd6-a964-2d649e921ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:23.695381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad1f86-5d49-4bd6-a964-2d649e921ccb admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:23.695668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad1f86-5d49-4bd6-a964-2d649e921ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:23.695950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad1f86-5d49-4bd6-a964-2d649e921ccb admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:23.696609 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad1f86-5d49-4bd6-a964-2d649e921ccb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:23.698637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad1f86-5d49-4bd6-a964-2d649e921ccb admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:23.894195 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1213/2438] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:52:23 2022] POST /identity/v3/projects => generated 383 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:52:23.902779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab1fc5-2b12-4aeb-8240-0abb2fbded9a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:23.914105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab1fc5-2b12-4aeb-8240-0abb2fbded9a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:24.064806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a4856-7f27-44d6-ba89-d7eeb64b8ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:24.065680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a4856-7f27-44d6-ba89-d7eeb64b8ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:24.066174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a4856-7f27-44d6-ba89-d7eeb64b8ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:24.066605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a4856-7f27-44d6-ba89-d7eeb64b8ffa admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:24.068309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a4856-7f27-44d6-ba89-d7eeb64b8ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:24.070508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a4856-7f27-44d6-ba89-d7eeb64b8ffa admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:24.250397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1226/2439] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:52:23 2022] POST /identity/v3/users => generated 292 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:52:24.250610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad1f86-5d49-4bd6-a964-2d649e921ccb admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:24.266179 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad1f86-5d49-4bd6-a964-2d649e921ccb admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:24.273960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f77d50-b713-4a5a-a700-60a295ce8228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:24.274993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f77d50-b713-4a5a-a700-60a295ce8228 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:24.277124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f77d50-b713-4a5a-a700-60a295ce8228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:24.278280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f77d50-b713-4a5a-a700-60a295ce8228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:24.279067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f77d50-b713-4a5a-a700-60a295ce8228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:24.280741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f77d50-b713-4a5a-a700-60a295ce8228 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:24.297927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1214/2440] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:52:24.303759 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a4856-7f27-44d6-ba89-d7eeb64b8ffa admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:24.316459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a4856-7f27-44d6-ba89-d7eeb64b8ffa admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:24.327799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d30731-04de-4100-a96f-453df5106abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:24.330383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d30731-04de-4100-a96f-453df5106abb admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:24.330928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d30731-04de-4100-a96f-453df5106abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:24.331226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d30731-04de-4100-a96f-453df5106abb admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:24.332227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d30731-04de-4100-a96f-453df5106abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:24.334545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d30731-04de-4100-a96f-453df5106abb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:24.352044 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1227/2441] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:52:24.356439 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f77d50-b713-4a5a-a700-60a295ce8228 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:24.370821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f77d50-b713-4a5a-a700-60a295ce8228 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:24.377475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df04e6-a28f-481b-b777-e93b04a35a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:24.378379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df04e6-a28f-481b-b777-e93b04a35a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:24.378729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df04e6-a28f-481b-b777-e93b04a35a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:24.379068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df04e6-a28f-481b-b777-e93b04a35a0b admin admin] PATH_INFO: `/v3/projects/96bff3017bdb43c1a46ce32757908067/users/cd51701678124941a632771175f63c4f/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:24.405367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df04e6-a28f-481b-b777-e93b04a35a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=96bff3017bdb43c1a46ce32757908067, user_id=cd51701678124941a632771175f63c4f, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:24.405367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df04e6-a28f-481b-b777-e93b04a35a0b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:24.510413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1215/2442] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:52:24 2022] PUT /identity/v3/projects/96bff3017bdb43c1a46ce32757908067/users/cd51701678124941a632771175f63c4f/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:52:24.521958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43a81-d3c7-4f82-81c1-dced8b555f61 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:24.522258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43a81-d3c7-4f82-81c1-dced8b555f61 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:24.522711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43a81-d3c7-4f82-81c1-dced8b555f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:24.559500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-31f43a81-d3c7-4f82-81c1-dced8b555f61 None None] MFA Rules not processed for user `cd51701678124941a632771175f63c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:24.560658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-31f43a81-d3c7-4f82-81c1-dced8b555f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:24.665378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1228/2443] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:24.670892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df04e6-a28f-481b-b777-e93b04a35a0b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:24.787398 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df04e6-a28f-481b-b777-e93b04a35a0b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:24.795446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812727c-7906-4990-b219-25a053863baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:24.796281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812727c-7906-4990-b219-25a053863baa admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:24.796561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812727c-7906-4990-b219-25a053863baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:24.796926 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812727c-7906-4990-b219-25a053863baa admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:24.797655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812727c-7906-4990-b219-25a053863baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:24.805031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812727c-7906-4990-b219-25a053863baa admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:24.827034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:24.903850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc4639-6c32-4637-87fc-0575305a4890 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:24.906403 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc4639-6c32-4637-87fc-0575305a4890 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:24.906682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc4639-6c32-4637-87fc-0575305a4890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:24.936469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1216/2444] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:52:24 2022] POST /identity/v3/projects => generated 385 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:52:24.938617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-55dc4639-6c32-4637-87fc-0575305a4890 None None] MFA Rules not processed for user `fb627e22492140858b12dcf4c0a70821`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:24.940423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812727c-7906-4990-b219-25a053863baa admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:24.942463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-55dc4639-6c32-4637-87fc-0575305a4890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:24.986922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:25.028321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1229/2445] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:52:25.038059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812727c-7906-4990-b219-25a053863baa admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:25.044856 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a4ac8-4271-4eea-9b44-d08206252019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:25.045578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a4ac8-4271-4eea-9b44-d08206252019 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:25.045962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a4ac8-4271-4eea-9b44-d08206252019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:25.046305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a4ac8-4271-4eea-9b44-d08206252019 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:25.047002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a4ac8-4271-4eea-9b44-d08206252019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:25.049988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a4ac8-4271-4eea-9b44-d08206252019 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:25.199272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1217/2446] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:52:24 2022] POST /identity/v3/users => generated 299 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:52:25.200480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc4639-6c32-4637-87fc-0575305a4890 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:25.212914 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc4639-6c32-4637-87fc-0575305a4890 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:25.220739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27bfd7d-dd7e-4dea-8a7a-918171ca304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:25.221353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bfd7d-dd7e-4dea-8a7a-918171ca304f admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:25.221630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bfd7d-dd7e-4dea-8a7a-918171ca304f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:25.221926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bfd7d-dd7e-4dea-8a7a-918171ca304f admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:25.222579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27bfd7d-dd7e-4dea-8a7a-918171ca304f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:25.225027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27bfd7d-dd7e-4dea-8a7a-918171ca304f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:25.238026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1230/2447] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:52:25.244020 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a4ac8-4271-4eea-9b44-d08206252019 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:25.254690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a4ac8-4271-4eea-9b44-d08206252019 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:25.259630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428a937-a29d-4243-97dd-93493ccec4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:25.260346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428a937-a29d-4243-97dd-93493ccec4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:25.260630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428a937-a29d-4243-97dd-93493ccec4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:25.260901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428a937-a29d-4243-97dd-93493ccec4c5 admin admin] PATH_INFO: `/v3/projects/e7666840c4794631a0c3f0a0480a3cfc/users/95a92eb1af314a66ae38a162ea564547/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:25.276817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428a937-a29d-4243-97dd-93493ccec4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7666840c4794631a0c3f0a0480a3cfc, user_id=95a92eb1af314a66ae38a162ea564547, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:25.279518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428a937-a29d-4243-97dd-93493ccec4c5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:25.375248 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1218/2448] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:52:25 2022] PUT /identity/v3/projects/e7666840c4794631a0c3f0a0480a3cfc/users/95a92eb1af314a66ae38a162ea564547/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:25.378422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27bfd7d-dd7e-4dea-8a7a-918171ca304f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:25.394870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27bfd7d-dd7e-4dea-8a7a-918171ca304f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:25.464207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e9d74-0973-4b18-a9b5-40669a97acc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:25.464999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e9d74-0973-4b18-a9b5-40669a97acc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:25.465135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e9d74-0973-4b18-a9b5-40669a97acc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:25.465508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e9d74-0973-4b18-a9b5-40669a97acc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:25.466357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e9d74-0973-4b18-a9b5-40669a97acc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:25.467984 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e9d74-0973-4b18-a9b5-40669a97acc8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:25.483600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1231/2449] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:52:25.484936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428a937-a29d-4243-97dd-93493ccec4c5 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:25.499203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428a937-a29d-4243-97dd-93493ccec4c5 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:25.504042 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab17d7-9756-4174-ac0f-b78567f3efb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:25.504858 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab17d7-9756-4174-ac0f-b78567f3efb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:25.505183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab17d7-9756-4174-ac0f-b78567f3efb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:25.505470 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab17d7-9756-4174-ac0f-b78567f3efb8 admin admin] PATH_INFO: `/v3/projects/e7666840c4794631a0c3f0a0480a3cfc/users/95a92eb1af314a66ae38a162ea564547/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:25.507689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab17d7-9756-4174-ac0f-b78567f3efb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7666840c4794631a0c3f0a0480a3cfc, user_id=95a92eb1af314a66ae38a162ea564547, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:25.510443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab17d7-9756-4174-ac0f-b78567f3efb8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:25.522911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1219/2450] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:52:25 2022] PUT /identity/v3/projects/e7666840c4794631a0c3f0a0480a3cfc/users/95a92eb1af314a66ae38a162ea564547/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:52:25.594826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85500ed5-274b-4a7e-905a-96a42f723a55 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:25.595072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85500ed5-274b-4a7e-905a-96a42f723a55 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:25.595400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85500ed5-274b-4a7e-905a-96a42f723a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:25.628865 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-85500ed5-274b-4a7e-905a-96a42f723a55 None None] MFA Rules not processed for user `95a92eb1af314a66ae38a162ea564547`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:25.630399 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-85500ed5-274b-4a7e-905a-96a42f723a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:25.755410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1232/2451] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:25.763861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab17d7-9756-4174-ac0f-b78567f3efb8 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:25.765797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a7ab17d7-9756-4174-ac0f-b78567f3efb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:25.856377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab17d7-9756-4174-ac0f-b78567f3efb8 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:25.861543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72439b-537f-421e-bc45-b237eda5a839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:25.862323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72439b-537f-421e-bc45-b237eda5a839 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:25.862673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72439b-537f-421e-bc45-b237eda5a839 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:25.862944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72439b-537f-421e-bc45-b237eda5a839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:25.957543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72439b-537f-421e-bc45-b237eda5a839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:25.959305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72439b-537f-421e-bc45-b237eda5a839 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:25.971792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1220/2452] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:27.306381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85500ed5-274b-4a7e-905a-96a42f723a55 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:27.307800 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-85500ed5-274b-4a7e-905a-96a42f723a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:27.368803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85500ed5-274b-4a7e-905a-96a42f723a55 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:27.374925 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9328d03-dfb5-42d1-8534-42926bf76b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:27.375557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9328d03-dfb5-42d1-8534-42926bf76b69 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:27.375854 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9328d03-dfb5-42d1-8534-42926bf76b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:27.376107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9328d03-dfb5-42d1-8534-42926bf76b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:27.378144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d9328d03-dfb5-42d1-8534-42926bf76b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:27.384692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:27.450164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9328d03-dfb5-42d1-8534-42926bf76b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:27.451602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9328d03-dfb5-42d1-8534-42926bf76b69 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:27.464999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1233/2453] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:28.254325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72439b-537f-421e-bc45-b237eda5a839 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:28.259800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:28.265356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72439b-537f-421e-bc45-b237eda5a839 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:28.271424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc66eef-8f11-411a-813f-fc8c3109da24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:28.272312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc66eef-8f11-411a-813f-fc8c3109da24 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:28.272746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc66eef-8f11-411a-813f-fc8c3109da24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:28.273232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc66eef-8f11-411a-813f-fc8c3109da24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:28.333846 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc66eef-8f11-411a-813f-fc8c3109da24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:28.335351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc66eef-8f11-411a-813f-fc8c3109da24 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:28.350737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1221/2454] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:30.111078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9328d03-dfb5-42d1-8534-42926bf76b69 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:30.124576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9328d03-dfb5-42d1-8534-42926bf76b69 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:30.129890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fc491-0183-46bf-9793-0cea0f715ab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:30.130735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc491-0183-46bf-9793-0cea0f715ab2 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:30.131225 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc491-0183-46bf-9793-0cea0f715ab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:30.131688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc491-0183-46bf-9793-0cea0f715ab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:30.144377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fc491-0183-46bf-9793-0cea0f715ab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:30.149983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fc491-0183-46bf-9793-0cea0f715ab2 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:30.162426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1234/2455] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:30.657899 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363508a-7c38-40aa-95b0-c865ee22cfef None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:30.658929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363508a-7c38-40aa-95b0-c865ee22cfef None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:30.659432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363508a-7c38-40aa-95b0-c865ee22cfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:30.704267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d363508a-7c38-40aa-95b0-c865ee22cfef None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:30.705764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d363508a-7c38-40aa-95b0-c865ee22cfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:30.799808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1222/2456] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:52:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:30.815055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fc491-0183-46bf-9793-0cea0f715ab2 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:30.816591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-959fc491-0183-46bf-9793-0cea0f715ab2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:30.916095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fc491-0183-46bf-9793-0cea0f715ab2 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:30.922625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd6f50-22df-47a2-8077-4c3c3b566d86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:30.923297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd6f50-22df-47a2-8077-4c3c3b566d86 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:30.923665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd6f50-22df-47a2-8077-4c3c3b566d86 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:30.923985 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd6f50-22df-47a2-8077-4c3c3b566d86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:30.938301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd6f50-22df-47a2-8077-4c3c3b566d86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:30.940222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd6f50-22df-47a2-8077-4c3c3b566d86 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:30.960448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1235/2457] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:52:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:52:35.324196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363508a-7c38-40aa-95b0-c865ee22cfef None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:35.395570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363508a-7c38-40aa-95b0-c865ee22cfef None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:35.400849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:35.401990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:35.402504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:35.402928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] PATH_INFO: `/v3/users/28476ef1072a4f998abfa3bcf0df3dd0` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:35.404510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28476ef1072a4f998abfa3bcf0df3dd0)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:35.407127 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:35.571177 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28476ef1072a4f998abfa3bcf0df3dd0'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:35.573064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:35.641389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28476ef1072a4f998abfa3bcf0df3dd0'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:35.641974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28476ef1072a4f998abfa3bcf0df3dd0'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:35.649980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28476ef1072a4f998abfa3bcf0df3dd0'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:35.667413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28476ef1072a4f998abfa3bcf0df3dd0'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:35.679308 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1223/2458] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:52:35 2022] DELETE /identity/v3/users/28476ef1072a4f998abfa3bcf0df3dd0 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:35.681013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0cfc8-d38a-42ad-bc72-3d02edb993fa None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:35.681493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0cfc8-d38a-42ad-bc72-3d02edb993fa None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:35.681591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0cfc8-d38a-42ad-bc72-3d02edb993fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:35.706903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-78c0cfc8-d38a-42ad-bc72-3d02edb993fa None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:35.708567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-78c0cfc8-d38a-42ad-bc72-3d02edb993fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:35.791301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1236/2459] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:35.801161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:35.803188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:35.871692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84bd74-0cae-4167-82be-7b76dedf93e6 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:35.875313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefefbf1-477d-47d8-885f-dff463f7e7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:35.875914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefefbf1-477d-47d8-885f-dff463f7e7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:35.876191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefefbf1-477d-47d8-885f-dff463f7e7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:35.876461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefefbf1-477d-47d8-885f-dff463f7e7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:35.885475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefefbf1-477d-47d8-885f-dff463f7e7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:35.886967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefefbf1-477d-47d8-885f-dff463f7e7a1 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:35.897013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1224/2460] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:36.398783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0cfc8-d38a-42ad-bc72-3d02edb993fa None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:36.400226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-78c0cfc8-d38a-42ad-bc72-3d02edb993fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:36.412864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0cfc8-d38a-42ad-bc72-3d02edb993fa None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:36.418803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:36.419455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:36.419737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:36.420014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] PATH_INFO: `/v3/projects/28fb779c268540e9b0d71bbffd80c32f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:36.421154 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=28fb779c268540e9b0d71bbffd80c32f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:36.423157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:36.437150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:36.551776 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28fb779c268540e9b0d71bbffd80c32f', 'request_id': 'req-40c17890-0cdb-4e28-b448-a4f8f6938e5f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:36.562183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28fb779c268540e9b0d71bbffd80c32f', 'request_id': 'req-40c17890-0cdb-4e28-b448-a4f8f6938e5f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:36.641486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28fb779c268540e9b0d71bbffd80c32f', 'request_id': 'req-40c17890-0cdb-4e28-b448-a4f8f6938e5f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:36.656025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] The token cache is being invalidate because project 28fb779c268540e9b0d71bbffd80c32f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:52:36.656429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:36.657157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:36.658881 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1237/2461] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:52:36 2022] DELETE /identity/v3/projects/28fb779c268540e9b0d71bbffd80c32f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:36.663315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefefbf1-477d-47d8-885f-dff463f7e7a1 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:36.673254 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefefbf1-477d-47d8-885f-dff463f7e7a1 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:36.742526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:36.743153 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:36.743533 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:36.743829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] PATH_INFO: `/v3/users/cd51701678124941a632771175f63c4f` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:36.745057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd51701678124941a632771175f63c4f)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:36.747043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:36.964586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd51701678124941a632771175f63c4f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:37.058828 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd51701678124941a632771175f63c4f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:37.059897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd51701678124941a632771175f63c4f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:37.069408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd51701678124941a632771175f63c4f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:37.075779 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:37.082694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd51701678124941a632771175f63c4f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:37.088648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1225/2462] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:52:36 2022] DELETE /identity/v3/users/cd51701678124941a632771175f63c4f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:52:37.558005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:37.559555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:37.576845 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c17890-0cdb-4e28-b448-a4f8f6938e5f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:37.644169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:37.644869 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:37.645187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:37.645504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] PATH_INFO: `/v3/projects/96bff3017bdb43c1a46ce32757908067` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:37.646730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96bff3017bdb43c1a46ce32757908067)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:37.648778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:37.768315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96bff3017bdb43c1a46ce32757908067', 'request_id': 'req-4c28ee61-06e0-44f7-904f-cf8910c71413'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:37.776130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96bff3017bdb43c1a46ce32757908067', 'request_id': 'req-4c28ee61-06e0-44f7-904f-cf8910c71413'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:37.839969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96bff3017bdb43c1a46ce32757908067', 'request_id': 'req-4c28ee61-06e0-44f7-904f-cf8910c71413'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:37.856530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] The token cache is being invalidate because project 96bff3017bdb43c1a46ce32757908067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:52:37.856849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:37.857715 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:37.859919 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1238/2463] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:52:37 2022] DELETE /identity/v3/projects/96bff3017bdb43c1a46ce32757908067 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:37.864455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:37.866204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:37.880930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b3f07-c7fd-45ca-9a68-bce144a80dde admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:37.955397 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:37.956044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:37.957972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:37.958278 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] PATH_INFO: `/v3/users/95a92eb1af314a66ae38a162ea564547` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:37.959654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=95a92eb1af314a66ae38a162ea564547)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:37.961969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:38.639478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95a92eb1af314a66ae38a162ea564547'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:38.710241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95a92eb1af314a66ae38a162ea564547'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:38.710910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95a92eb1af314a66ae38a162ea564547'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:38.719615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95a92eb1af314a66ae38a162ea564547'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:38.734033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6ae0231d-ca1a-4569-9d56-3c4f2cea5e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95a92eb1af314a66ae38a162ea564547'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:38.740974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1226/2464] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:52:37 2022] DELETE /identity/v3/users/95a92eb1af314a66ae38a162ea564547 => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:39.250302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:39.252571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:39.270692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28ee61-06e0-44f7-904f-cf8910c71413 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:39.317627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:39.349172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:39.349933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:39.350280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:39.350617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] PATH_INFO: `/v3/projects/e7666840c4794631a0c3f0a0480a3cfc` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:39.351829 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7666840c4794631a0c3f0a0480a3cfc)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:39.353848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:39.466082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7666840c4794631a0c3f0a0480a3cfc', 'request_id': 'req-b3193eb4-aa19-44d4-9961-7468c0019176'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:39.473828 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7666840c4794631a0c3f0a0480a3cfc', 'request_id': 'req-b3193eb4-aa19-44d4-9961-7468c0019176'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:39.573031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7666840c4794631a0c3f0a0480a3cfc', 'request_id': 'req-b3193eb4-aa19-44d4-9961-7468c0019176'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:39.586511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] The token cache is being invalidate because project e7666840c4794631a0c3f0a0480a3cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:52:39.586935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:39.587779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3193eb4-aa19-44d4-9961-7468c0019176 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:39.590273 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1239/2465] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:52:39 2022] DELETE /identity/v3/projects/e7666840c4794631a0c3f0a0480a3cfc => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:39.599975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2b1c-0456-4480-997d-a0a6e3d2dbda None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:39.600615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2b1c-0456-4480-997d-a0a6e3d2dbda None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:39.600615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2b1c-0456-4480-997d-a0a6e3d2dbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:39.634370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-01fe2b1c-0456-4480-997d-a0a6e3d2dbda None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:39.635726 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-01fe2b1c-0456-4480-997d-a0a6e3d2dbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:39.732878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1227/2466] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:39.741988 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cece4d-4570-4a35-9d1c-a38037e0d25c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:39.742340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cece4d-4570-4a35-9d1c-a38037e0d25c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:39.742623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cece4d-4570-4a35-9d1c-a38037e0d25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:39.763321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-77cece4d-4570-4a35-9d1c-a38037e0d25c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:39.764533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-77cece4d-4570-4a35-9d1c-a38037e0d25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:39.773661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1240/2467] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:39.777849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe2b1c-0456-4480-997d-a0a6e3d2dbda None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:39.790805 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe2b1c-0456-4480-997d-a0a6e3d2dbda None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:39.814236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6a2d8-d00d-4e0c-9bb8-f1c82be87824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:39.815009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6a2d8-d00d-4e0c-9bb8-f1c82be87824 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:39.815406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6a2d8-d00d-4e0c-9bb8-f1c82be87824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:39.815780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6a2d8-d00d-4e0c-9bb8-f1c82be87824 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:39.816543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6a2d8-d00d-4e0c-9bb8-f1c82be87824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:52:39.817019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6a2d8-d00d-4e0c-9bb8-f1c82be87824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:39.819317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6a2d8-d00d-4e0c-9bb8-f1c82be87824 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:39.821361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:39.845232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cece4d-4570-4a35-9d1c-a38037e0d25c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:39.845855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1228/2468] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:52:39.855466 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cece4d-4570-4a35-9d1c-a38037e0d25c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:39.860589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6d1cf-c384-4991-9cd2-84e2a9f254a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:39.861203 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6d1cf-c384-4991-9cd2-84e2a9f254a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:39.861513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6d1cf-c384-4991-9cd2-84e2a9f254a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:39.861918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6d1cf-c384-4991-9cd2-84e2a9f254a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:39.862577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6d1cf-c384-4991-9cd2-84e2a9f254a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:39.865036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6d1cf-c384-4991-9cd2-84e2a9f254a3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:40.038591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1241/2469] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:52:39 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:52:40.042294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6a2d8-d00d-4e0c-9bb8-f1c82be87824 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:40.060659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6a2d8-d00d-4e0c-9bb8-f1c82be87824 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:40.122345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad709b3-aad3-4d7b-b5c2-b5f99d3325f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:40.122867 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad709b3-aad3-4d7b-b5c2-b5f99d3325f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:40.123150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad709b3-aad3-4d7b-b5c2-b5f99d3325f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:40.123425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad709b3-aad3-4d7b-b5c2-b5f99d3325f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:40.124068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad709b3-aad3-4d7b-b5c2-b5f99d3325f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:40.126044 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad709b3-aad3-4d7b-b5c2-b5f99d3325f9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:40.271607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1229/2470] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:52:40 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:52:40.275762 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6d1cf-c384-4991-9cd2-84e2a9f254a3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:40.287297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6d1cf-c384-4991-9cd2-84e2a9f254a3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:40.292457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab377411-c617-45e5-b631-edba92c65969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:40.293148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab377411-c617-45e5-b631-edba92c65969 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:40.293502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab377411-c617-45e5-b631-edba92c65969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:40.293888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab377411-c617-45e5-b631-edba92c65969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:40.294614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab377411-c617-45e5-b631-edba92c65969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:40.296296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab377411-c617-45e5-b631-edba92c65969 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:40.308407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1242/2471] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:52:40.312496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad709b3-aad3-4d7b-b5c2-b5f99d3325f9 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:40.323517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad709b3-aad3-4d7b-b5c2-b5f99d3325f9 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:40.328099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48e030-bb9f-4868-9320-ad03aadfb3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:40.328695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48e030-bb9f-4868-9320-ad03aadfb3df admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:40.329011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48e030-bb9f-4868-9320-ad03aadfb3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:40.329276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48e030-bb9f-4868-9320-ad03aadfb3df admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:40.329957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48e030-bb9f-4868-9320-ad03aadfb3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:40.332924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48e030-bb9f-4868-9320-ad03aadfb3df admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:40.343980 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1230/2472] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:52:40.349244 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab377411-c617-45e5-b631-edba92c65969 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:40.361290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab377411-c617-45e5-b631-edba92c65969 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:40.367051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce442be9-395a-4fa5-9be5-fdac56045296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:40.367801 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce442be9-395a-4fa5-9be5-fdac56045296 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:40.368095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce442be9-395a-4fa5-9be5-fdac56045296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:40.368446 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce442be9-395a-4fa5-9be5-fdac56045296 admin admin] PATH_INFO: `/v3/projects/d50d14c9714542a1baa271ba21f3ad30/users/7f4f0fa46dde45f6ad0ebc9f189a10c9/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:40.389234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce442be9-395a-4fa5-9be5-fdac56045296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50d14c9714542a1baa271ba21f3ad30, user_id=7f4f0fa46dde45f6ad0ebc9f189a10c9, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:40.392529 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce442be9-395a-4fa5-9be5-fdac56045296 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:40.499975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1243/2473] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:52:40 2022] PUT /identity/v3/projects/d50d14c9714542a1baa271ba21f3ad30/users/7f4f0fa46dde45f6ad0ebc9f189a10c9/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:40.513849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a582fe-b8d2-41cb-b539-c35b6398ffea None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:40.513849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a582fe-b8d2-41cb-b539-c35b6398ffea None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:40.513849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a582fe-b8d2-41cb-b539-c35b6398ffea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:40.560886 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-85a582fe-b8d2-41cb-b539-c35b6398ffea None None] MFA Rules not processed for user `7f4f0fa46dde45f6ad0ebc9f189a10c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:40.563577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-85a582fe-b8d2-41cb-b539-c35b6398ffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:40.642483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1231/2474] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:40.651405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce442be9-395a-4fa5-9be5-fdac56045296 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:40.653197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ce442be9-395a-4fa5-9be5-fdac56045296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:40.715236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce442be9-395a-4fa5-9be5-fdac56045296 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:40.719917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd96b85-40b4-4343-a8cf-c33eded69d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:40.720618 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd96b85-40b4-4343-a8cf-c33eded69d8e service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:40.720815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd96b85-40b4-4343-a8cf-c33eded69d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:40.721090 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd96b85-40b4-4343-a8cf-c33eded69d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:40.733966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd96b85-40b4-4343-a8cf-c33eded69d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:40.733966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd96b85-40b4-4343-a8cf-c33eded69d8e service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:40.737296 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:40.742196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1244/2475] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:44.495799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a582fe-b8d2-41cb-b539-c35b6398ffea None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:44.558762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a582fe-b8d2-41cb-b539-c35b6398ffea None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:44.563588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:44.564229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:44.564490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:44.564764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] PATH_INFO: `/v3/users/7f4f0fa46dde45f6ad0ebc9f189a10c9` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:44.565930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4f0fa46dde45f6ad0ebc9f189a10c9)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:44.568102 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:44.578140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:44.852001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4f0fa46dde45f6ad0ebc9f189a10c9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:44.928502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4f0fa46dde45f6ad0ebc9f189a10c9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:44.929153 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4f0fa46dde45f6ad0ebc9f189a10c9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:44.937483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4f0fa46dde45f6ad0ebc9f189a10c9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:44.949972 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4f0fa46dde45f6ad0ebc9f189a10c9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:44.956040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1232/2476] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:52:44 2022] DELETE /identity/v3/users/7f4f0fa46dde45f6ad0ebc9f189a10c9 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:44.962054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55fbe1-5503-48fa-a8e8-946b8ce733e1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:44.962725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55fbe1-5503-48fa-a8e8-946b8ce733e1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:44.962725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55fbe1-5503-48fa-a8e8-946b8ce733e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:44.984879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6f55fbe1-5503-48fa-a8e8-946b8ce733e1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:44.986109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6f55fbe1-5503-48fa-a8e8-946b8ce733e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:45.050228 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1245/2477] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:45.060627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:45.062016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:45.122046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dac31-8811-4ea9-88f8-9265c8bdff27 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:45.125549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2906a29-3045-46eb-85dd-368ffd7270d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:45.126181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906a29-3045-46eb-85dd-368ffd7270d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:45.126461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906a29-3045-46eb-85dd-368ffd7270d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:45.126732 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906a29-3045-46eb-85dd-368ffd7270d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:45.135954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2906a29-3045-46eb-85dd-368ffd7270d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:45.137370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2906a29-3045-46eb-85dd-368ffd7270d9 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:45.148197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1233/2478] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:52:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:52:45.696356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55fbe1-5503-48fa-a8e8-946b8ce733e1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:52:45.698340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6f55fbe1-5503-48fa-a8e8-946b8ce733e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:45.712937 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55fbe1-5503-48fa-a8e8-946b8ce733e1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:52:45.725002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:52:45.726364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:45.726736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:45.727187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] PATH_INFO: `/v3/projects/d50d14c9714542a1baa271ba21f3ad30` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:45.730630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50d14c9714542a1baa271ba21f3ad30)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:52:45.736095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:52:45.928596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50d14c9714542a1baa271ba21f3ad30', 'request_id': 'req-a33d53cb-6313-4f2a-889b-51eb2d06d096'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:45.935038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50d14c9714542a1baa271ba21f3ad30', 'request_id': 'req-a33d53cb-6313-4f2a-889b-51eb2d06d096'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:46.029595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50d14c9714542a1baa271ba21f3ad30', 'request_id': 'req-a33d53cb-6313-4f2a-889b-51eb2d06d096'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:46.041503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] The token cache is being invalidate because project d50d14c9714542a1baa271ba21f3ad30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:52:46.041819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:46.043127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:52:46.044092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1246/2479] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:52:45 2022] DELETE /identity/v3/projects/d50d14c9714542a1baa271ba21f3ad30 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:52:55.142587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:52:55.808537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0646d-4bc4-4fd3-b38b-785abed3b096 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:52:55.808817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0646d-4bc4-4fd3-b38b-785abed3b096 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:52:55.809088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0646d-4bc4-4fd3-b38b-785abed3b096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:52:55.828857 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-d8d0646d-4bc4-4fd3-b38b-785abed3b096 None None] MFA Rules not processed for user `fafa609d94334db6907b401e66cb3448`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:52:55.830889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d8d0646d-4bc4-4fd3-b38b-785abed3b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:52:55.887818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1234/2480] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:52:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:52:56.038893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:53:05.913704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=25 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:53:06.652348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:53:06.653984 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:53:06.725683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d53cb-6313-4f2a-889b-51eb2d06d096 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:53:06.729440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a05ff-ec4f-4146-88cc-afee650b15bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:53:06.730081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a05ff-ec4f-4146-88cc-afee650b15bf service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:53:06.730363 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a05ff-ec4f-4146-88cc-afee650b15bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:53:06.731009 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a05ff-ec4f-4146-88cc-afee650b15bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:53:06.733336 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5a3a05ff-ec4f-4146-88cc-afee650b15bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:53:06.806058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a05ff-ec4f-4146-88cc-afee650b15bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:53:06.807621 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a05ff-ec4f-4146-88cc-afee650b15bf service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:53:06.821232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1247/2481] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:53:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:53:12.494266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0646d-4bc4-4fd3-b38b-785abed3b096 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:53:12.496537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d8d0646d-4bc4-4fd3-b38b-785abed3b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:53:12.516276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0646d-4bc4-4fd3-b38b-785abed3b096 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:53:12.522844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e86f3-b28e-47f6-a741-d6ac30deefc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:53:12.523624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e86f3-b28e-47f6-a741-d6ac30deefc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:53:12.523907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e86f3-b28e-47f6-a741-d6ac30deefc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:53:12.524220 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e86f3-b28e-47f6-a741-d6ac30deefc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:53:12.526683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-631e86f3-b28e-47f6-a741-d6ac30deefc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:53:12.608821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e86f3-b28e-47f6-a741-d6ac30deefc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:53:12.610294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e86f3-b28e-47f6-a741-d6ac30deefc4 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:53:12.622405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1235/2482] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:53:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:53:16.814284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=50 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:53:20.597603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e824-7c55-4d46-b460-6de8f9770141 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:53:20.598907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e824-7c55-4d46-b460-6de8f9770141 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:53:20.598907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e824-7c55-4d46-b460-6de8f9770141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:53:20.618693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c7d7e824-7c55-4d46-b460-6de8f9770141 None None] MFA Rules not processed for user `2723e5c74bde49f485099b615b36997a`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:53:20.623868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c7d7e824-7c55-4d46-b460-6de8f9770141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:53:20.674849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1248/2483] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:53:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:53:21.280592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ae8bb-1795-4ee6-b76a-fa09260b92a3 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:53:21.281105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ae8bb-1795-4ee6-b76a-fa09260b92a3 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:53:21.281219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ae8bb-1795-4ee6-b76a-fa09260b92a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:53:21.300896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-048ae8bb-1795-4ee6-b76a-fa09260b92a3 None None] MFA Rules not processed for user `26995363963a4a52acc3f57ce1575e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:53:21.302169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-048ae8bb-1795-4ee6-b76a-fa09260b92a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:53:21.364998 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1236/2484] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:53:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:53:30.668022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=25 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:53:31.351599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=55 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:53:41.057380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e824-7c55-4d46-b460-6de8f9770141 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:53:41.068678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e824-7c55-4d46-b460-6de8f9770141 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:53:41.072364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6f845-2a78-4131-9821-dc7586a39514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:53:41.072978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6f845-2a78-4131-9821-dc7586a39514 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:53:41.073258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6f845-2a78-4131-9821-dc7586a39514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:53:41.073532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6f845-2a78-4131-9821-dc7586a39514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:53:41.075321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6ad6f845-2a78-4131-9821-dc7586a39514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:53:41.156229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6f845-2a78-4131-9821-dc7586a39514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:53:41.157433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6f845-2a78-4131-9821-dc7586a39514 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:53:41.169320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1249/2485] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:53:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:53:42.370485 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ae8bb-1795-4ee6-b76a-fa09260b92a3 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:53:42.380704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ae8bb-1795-4ee6-b76a-fa09260b92a3 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:53:42.384304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ea4c7-1ef1-4b65-936d-44f56c83a153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:53:42.384893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ea4c7-1ef1-4b65-936d-44f56c83a153 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:53:42.385169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ea4c7-1ef1-4b65-936d-44f56c83a153 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:53:42.385434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ea4c7-1ef1-4b65-936d-44f56c83a153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:53:42.387176 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-470ea4c7-1ef1-4b65-936d-44f56c83a153 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:53:42.400918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ea4c7-1ef1-4b65-936d-44f56c83a153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:53:42.402403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ea4c7-1ef1-4b65-936d-44f56c83a153 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:53:42.412986 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1237/2486] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:53:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:53:51.164072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=34 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:53:52.408287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=6 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:54:03.828719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6f845-2a78-4131-9821-dc7586a39514 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:03.830171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6ad6f845-2a78-4131-9821-dc7586a39514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:03.844758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6f845-2a78-4131-9821-dc7586a39514 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:03.850416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85b418-3f6f-4d2c-a1f6-045ed73fd8dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:03.851022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85b418-3f6f-4d2c-a1f6-045ed73fd8dc service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:03.851299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85b418-3f6f-4d2c-a1f6-045ed73fd8dc service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:03.851570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85b418-3f6f-4d2c-a1f6-045ed73fd8dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:03.853263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6b85b418-3f6f-4d2c-a1f6-045ed73fd8dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:03.864728 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85b418-3f6f-4d2c-a1f6-045ed73fd8dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:03.866390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85b418-3f6f-4d2c-a1f6-045ed73fd8dc service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:03.875639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1250/2487] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:08.808672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ea4c7-1ef1-4b65-936d-44f56c83a153 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:08.810215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-470ea4c7-1ef1-4b65-936d-44f56c83a153 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:08.868060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ea4c7-1ef1-4b65-936d-44f56c83a153 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:08.871624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650c461-00a1-4d9f-b5f1-51d0e86332b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:08.872221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650c461-00a1-4d9f-b5f1-51d0e86332b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:08.872513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650c461-00a1-4d9f-b5f1-51d0e86332b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:08.872763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650c461-00a1-4d9f-b5f1-51d0e86332b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:08.874496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e650c461-00a1-4d9f-b5f1-51d0e86332b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:08.887140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650c461-00a1-4d9f-b5f1-51d0e86332b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:08.889219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650c461-00a1-4d9f-b5f1-51d0e86332b6 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:08.898472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1238/2488] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:13.871364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:54:18.894116 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=26 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:54:23.934880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85b418-3f6f-4d2c-a1f6-045ed73fd8dc service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:23.947794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85b418-3f6f-4d2c-a1f6-045ed73fd8dc service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:23.953564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f2278-4d23-4f14-ad81-7577840c29ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:23.954204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2278-4d23-4f14-ad81-7577840c29ea service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:23.954664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2278-4d23-4f14-ad81-7577840c29ea service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:23.954943 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2278-4d23-4f14-ad81-7577840c29ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:23.956726 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6b5f2278-4d23-4f14-ad81-7577840c29ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:23.971369 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f2278-4d23-4f14-ad81-7577840c29ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:23.973104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f2278-4d23-4f14-ad81-7577840c29ea service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:23.985855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1251/2489] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:24.970394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650c461-00a1-4d9f-b5f1-51d0e86332b6 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:24.983612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650c461-00a1-4d9f-b5f1-51d0e86332b6 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:24.988989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381cf58-ce46-4be0-b3ad-a2c3c416cc82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:24.989584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381cf58-ce46-4be0-b3ad-a2c3c416cc82 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:24.989900 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381cf58-ce46-4be0-b3ad-a2c3c416cc82 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:24.990218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381cf58-ce46-4be0-b3ad-a2c3c416cc82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:24.992048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7381cf58-ce46-4be0-b3ad-a2c3c416cc82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:25.006412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381cf58-ce46-4be0-b3ad-a2c3c416cc82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:25.008658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381cf58-ce46-4be0-b3ad-a2c3c416cc82 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:25.019779 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1239/2490] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:29.122893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c772c-0cd4-4b35-a0e5-a62e15ac3fe7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:29.123381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c772c-0cd4-4b35-a0e5-a62e15ac3fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:29.123519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c772c-0cd4-4b35-a0e5-a62e15ac3fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:29.142518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6b3c772c-0cd4-4b35-a0e5-a62e15ac3fe7 None None] MFA Rules not processed for user `fafa609d94334db6907b401e66cb3448`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:54:29.143700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6b3c772c-0cd4-4b35-a0e5-a62e15ac3fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:29.150862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1252/2491] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:54:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:54:29.462237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381cf58-ce46-4be0-b3ad-a2c3c416cc82 service nova] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:29.471567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381cf58-ce46-4be0-b3ad-a2c3c416cc82 service nova] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:29.476080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5222b73-be12-4b26-8605-f4235ed42106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:29.476659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5222b73-be12-4b26-8605-f4235ed42106 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:29.476939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5222b73-be12-4b26-8605-f4235ed42106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:29.477221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5222b73-be12-4b26-8605-f4235ed42106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:29.478950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c5222b73-be12-4b26-8605-f4235ed42106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:29.488793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5222b73-be12-4b26-8605-f4235ed42106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:29.490316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5222b73-be12-4b26-8605-f4235ed42106 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:29.498785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1240/2492] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:29.998449 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c772c-0cd4-4b35-a0e5-a62e15ac3fe7 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:30.000080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6b3c772c-0cd4-4b35-a0e5-a62e15ac3fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:30.071692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c772c-0cd4-4b35-a0e5-a62e15ac3fe7 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:30.076613 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbf6d0-9aab-460e-81e1-1f4fa18e3af7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ec5c9c71d14762a4869b7d66e87ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:30.077219 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbf6d0-9aab-460e-81e1-1f4fa18e3af7 service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:30.077497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbf6d0-9aab-460e-81e1-1f4fa18e3af7 service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:30.077795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbf6d0-9aab-460e-81e1-1f4fa18e3af7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:30.079522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-45cbf6d0-9aab-460e-81e1-1f4fa18e3af7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:30.091327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbf6d0-9aab-460e-81e1-1f4fa18e3af7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:30.093130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbf6d0-9aab-460e-81e1-1f4fa18e3af7 service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:30.103605 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1253/2493] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:33.501337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5222b73-be12-4b26-8605-f4235ed42106 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:33.513414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5222b73-be12-4b26-8605-f4235ed42106 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:33.517829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54339f8e-a11f-4da8-bac7-576a94a502e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:33.518460 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54339f8e-a11f-4da8-bac7-576a94a502e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:33.518740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54339f8e-a11f-4da8-bac7-576a94a502e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:33.519013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54339f8e-a11f-4da8-bac7-576a94a502e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:33.520792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-54339f8e-a11f-4da8-bac7-576a94a502e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:33.531983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54339f8e-a11f-4da8-bac7-576a94a502e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:33.533410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54339f8e-a11f-4da8-bac7-576a94a502e7 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:33.542125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1241/2494] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:37.637558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbf6d0-9aab-460e-81e1-1f4fa18e3af7 service placement] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:37.659258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbf6d0-9aab-460e-81e1-1f4fa18e3af7 service placement] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:37.665961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21a4fd-0a68-41ad-8c49-bad611e3fd4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:37.667604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21a4fd-0a68-41ad-8c49-bad611e3fd4c service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:37.667604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21a4fd-0a68-41ad-8c49-bad611e3fd4c service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:37.667843 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21a4fd-0a68-41ad-8c49-bad611e3fd4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:37.669579 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3a21a4fd-0a68-41ad-8c49-bad611e3fd4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:37.684646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21a4fd-0a68-41ad-8c49-bad611e3fd4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:37.689997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21a4fd-0a68-41ad-8c49-bad611e3fd4c service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:37.708693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1254/2495] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:39.503741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54339f8e-a11f-4da8-bac7-576a94a502e7 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:39.516180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54339f8e-a11f-4da8-bac7-576a94a502e7 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:39.520751 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c7217-8987-4a53-b1c0-45557e1ee7bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:39.522236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c7217-8987-4a53-b1c0-45557e1ee7bd service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:39.522550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c7217-8987-4a53-b1c0-45557e1ee7bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:39.522870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c7217-8987-4a53-b1c0-45557e1ee7bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:39.524929 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1a7c7217-8987-4a53-b1c0-45557e1ee7bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:39.613130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c7217-8987-4a53-b1c0-45557e1ee7bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:39.614699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c7217-8987-4a53-b1c0-45557e1ee7bd service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:39.629025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1242/2496] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:42.124685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21a4fd-0a68-41ad-8c49-bad611e3fd4c service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:42.126254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3a21a4fd-0a68-41ad-8c49-bad611e3fd4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:42.142002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21a4fd-0a68-41ad-8c49-bad611e3fd4c service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:42.148523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31960d-5ed3-4274-922b-e3f1a39e1518 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:42.149433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31960d-5ed3-4274-922b-e3f1a39e1518 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:42.149950 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31960d-5ed3-4274-922b-e3f1a39e1518 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:42.150432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31960d-5ed3-4274-922b-e3f1a39e1518 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:42.152539 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0e31960d-5ed3-4274-922b-e3f1a39e1518 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:42.165941 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31960d-5ed3-4274-922b-e3f1a39e1518 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:42.168237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31960d-5ed3-4274-922b-e3f1a39e1518 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:42.199532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1255/2497] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:44.326495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c7217-8987-4a53-b1c0-45557e1ee7bd service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:44.328590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1a7c7217-8987-4a53-b1c0-45557e1ee7bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:44.344652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c7217-8987-4a53-b1c0-45557e1ee7bd service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:44.354392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:44.355118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:44.355401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:44.355681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] PATH_INFO: `/v3/users/26995363963a4a52acc3f57ce1575e6e` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:44.356842 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=26995363963a4a52acc3f57ce1575e6e)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:44.361179 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:44.585299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26995363963a4a52acc3f57ce1575e6e'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:54:44.656731 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26995363963a4a52acc3f57ce1575e6e'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:54:44.657985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26995363963a4a52acc3f57ce1575e6e'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:54:44.665568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26995363963a4a52acc3f57ce1575e6e'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:54:44.679265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-35dcf7e9-7aec-47dd-9c76-52864aa103ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26995363963a4a52acc3f57ce1575e6e'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:54:44.686574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1243/2498] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:54:44 2022] DELETE /identity/v3/users/26995363963a4a52acc3f57ce1575e6e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:54:44.955562 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31960d-5ed3-4274-922b-e3f1a39e1518 service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:44.966975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31960d-5ed3-4274-922b-e3f1a39e1518 service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:45.030686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:45.031320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:45.031704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:45.031984 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] PATH_INFO: `/v3/projects/294abb2085b844a99908242d4b02e263` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:45.033121 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=294abb2085b844a99908242d4b02e263)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:45.035057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:45.468496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:54:45.471209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '294abb2085b844a99908242d4b02e263', 'request_id': 'req-ffba589c-077f-4844-876f-55ab034828ff'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:54:45.477545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '294abb2085b844a99908242d4b02e263', 'request_id': 'req-ffba589c-077f-4844-876f-55ab034828ff'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:54:45.559233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '294abb2085b844a99908242d4b02e263', 'request_id': 'req-ffba589c-077f-4844-876f-55ab034828ff'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:54:45.570789 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] The token cache is being invalidate because project 294abb2085b844a99908242d4b02e263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:54:45.571079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:54:45.571713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ffba589c-077f-4844-876f-55ab034828ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:54:45.573204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1256/2499] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:54:44 2022] DELETE /identity/v3/projects/294abb2085b844a99908242d4b02e263 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:54:45.578154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a2a66-6f61-42fd-a8a2-f750f1b5303e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:45.578646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a2a66-6f61-42fd-a8a2-f750f1b5303e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:45.578757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a2a66-6f61-42fd-a8a2-f750f1b5303e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:45.599249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-837a2a66-6f61-42fd-a8a2-f750f1b5303e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:54:45.600470 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-837a2a66-6f61-42fd-a8a2-f750f1b5303e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:45.619863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:54:45.658990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1244/2500] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:54:45.666650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102cdf-375c-4874-9f1d-5853df0fd108 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:45.666939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102cdf-375c-4874-9f1d-5853df0fd108 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:45.667218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102cdf-375c-4874-9f1d-5853df0fd108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:45.687153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-65102cdf-375c-4874-9f1d-5853df0fd108 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:54:45.688356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-65102cdf-375c-4874-9f1d-5853df0fd108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:45.696944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1257/2501] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:54:45.704688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a2a66-6f61-42fd-a8a2-f750f1b5303e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:45.714936 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a2a66-6f61-42fd-a8a2-f750f1b5303e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:45.720191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50419332-5646-4ed9-8106-acc693e6b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:45.720786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50419332-5646-4ed9-8106-acc693e6b280 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:45.721068 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50419332-5646-4ed9-8106-acc693e6b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:45.721336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50419332-5646-4ed9-8106-acc693e6b280 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:45.722012 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50419332-5646-4ed9-8106-acc693e6b280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:54:45.722314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50419332-5646-4ed9-8106-acc693e6b280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:45.723935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50419332-5646-4ed9-8106-acc693e6b280 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:45.735945 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1245/2502] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:54:45.741152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102cdf-375c-4874-9f1d-5853df0fd108 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:45.750727 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102cdf-375c-4874-9f1d-5853df0fd108 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:45.755458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62613be7-5afa-4d5c-9451-b8e026362db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:45.757237 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62613be7-5afa-4d5c-9451-b8e026362db2 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:45.757515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62613be7-5afa-4d5c-9451-b8e026362db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:45.757819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62613be7-5afa-4d5c-9451-b8e026362db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:45.758465 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62613be7-5afa-4d5c-9451-b8e026362db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:45.760373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62613be7-5afa-4d5c-9451-b8e026362db2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:45.861324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1258/2503] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:54:45 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:54:45.862616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50419332-5646-4ed9-8106-acc693e6b280 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:45.874966 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50419332-5646-4ed9-8106-acc693e6b280 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:45.946941 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8807c-8020-4946-8c41-224d35dd265d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:45.948890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8807c-8020-4946-8c41-224d35dd265d admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:45.948890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8807c-8020-4946-8c41-224d35dd265d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:45.948890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8807c-8020-4946-8c41-224d35dd265d admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:45.948890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8807c-8020-4946-8c41-224d35dd265d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:45.950897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8807c-8020-4946-8c41-224d35dd265d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:46.098067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1246/2504] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:54:45 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:54:46.100279 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62613be7-5afa-4d5c-9451-b8e026362db2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:46.114138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62613be7-5afa-4d5c-9451-b8e026362db2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:46.133630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ab7c8-eded-4261-83d4-0631c1274e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:46.134751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab7c8-eded-4261-83d4-0631c1274e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:46.135138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab7c8-eded-4261-83d4-0631c1274e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:46.135525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab7c8-eded-4261-83d4-0631c1274e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:46.136269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ab7c8-eded-4261-83d4-0631c1274e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:46.138024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ab7c8-eded-4261-83d4-0631c1274e8f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:46.150347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1259/2505] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:54:46.154617 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8807c-8020-4946-8c41-224d35dd265d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:46.168902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8807c-8020-4946-8c41-224d35dd265d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:46.174688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c9689-c67a-44c5-a7ca-028cb07d63cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:46.175813 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c9689-c67a-44c5-a7ca-028cb07d63cc admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:46.176305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c9689-c67a-44c5-a7ca-028cb07d63cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:46.176645 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c9689-c67a-44c5-a7ca-028cb07d63cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:46.177359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c9689-c67a-44c5-a7ca-028cb07d63cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:46.179945 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c9689-c67a-44c5-a7ca-028cb07d63cc admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:46.199182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1247/2506] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:54:46.202860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ab7c8-eded-4261-83d4-0631c1274e8f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:46.219488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ab7c8-eded-4261-83d4-0631c1274e8f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:46.225974 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cee9da-5b3d-4aa3-9e29-293b7c12acd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:46.228129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cee9da-5b3d-4aa3-9e29-293b7c12acd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:46.228456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cee9da-5b3d-4aa3-9e29-293b7c12acd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:46.228936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cee9da-5b3d-4aa3-9e29-293b7c12acd8 admin admin] PATH_INFO: `/v3/projects/ddf27ec567494d03bd1d2469aef97264/users/3f46252462244a0a84180c4726d15f94/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:46.246797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cee9da-5b3d-4aa3-9e29-293b7c12acd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf27ec567494d03bd1d2469aef97264, user_id=3f46252462244a0a84180c4726d15f94, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:46.249831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cee9da-5b3d-4aa3-9e29-293b7c12acd8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:46.337540 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1260/2507] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:54:46 2022] PUT /identity/v3/projects/ddf27ec567494d03bd1d2469aef97264/users/3f46252462244a0a84180c4726d15f94/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:54:46.343607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30688554-af27-4905-bcad-7965a1d68e4b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:46.343908 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30688554-af27-4905-bcad-7965a1d68e4b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:46.344396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30688554-af27-4905-bcad-7965a1d68e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:46.368266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-30688554-af27-4905-bcad-7965a1d68e4b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:54:46.369464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-30688554-af27-4905-bcad-7965a1d68e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:46.476442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1248/2508] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:54:46.488231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cee9da-5b3d-4aa3-9e29-293b7c12acd8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:46.497115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c2cee9da-5b3d-4aa3-9e29-293b7c12acd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:46.594437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cee9da-5b3d-4aa3-9e29-293b7c12acd8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:46.600256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844dfad-0123-4c11-85d8-4c006bebf229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:46.600915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844dfad-0123-4c11-85d8-4c006bebf229 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:46.601460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844dfad-0123-4c11-85d8-4c006bebf229 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:46.601748 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844dfad-0123-4c11-85d8-4c006bebf229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:46.611451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844dfad-0123-4c11-85d8-4c006bebf229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:46.613031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844dfad-0123-4c11-85d8-4c006bebf229 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:46.624030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1261/2509] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:49.992885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296d7f-c44d-4526-99d3-f8ceafc034ae None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:49.992885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296d7f-c44d-4526-99d3-f8ceafc034ae None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:49.992885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296d7f-c44d-4526-99d3-f8ceafc034ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:50.005286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55995c53-1908-473a-886b-760191df069f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:50.006698 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55995c53-1908-473a-886b-760191df069f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:50.006698 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55995c53-1908-473a-886b-760191df069f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:50.023263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ed296d7f-c44d-4526-99d3-f8ceafc034ae None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:54:50.024515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ed296d7f-c44d-4526-99d3-f8ceafc034ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:50.043541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-55995c53-1908-473a-886b-760191df069f None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:54:50.043541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-55995c53-1908-473a-886b-760191df069f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:50.092912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:54:50.146209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1262/2510] 10.222.0.42 () {58 vars in 967 bytes} [Sat Sep 17 03:54:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:54:50.155994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1249/2511] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:54:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:54:50.194228 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55995c53-1908-473a-886b-760191df069f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:50.201193 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-55995c53-1908-473a-886b-760191df069f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:50.210338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296d7f-c44d-4526-99d3-f8ceafc034ae None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:50.211791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ed296d7f-c44d-4526-99d3-f8ceafc034ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:50.255007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:54:50.332058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55995c53-1908-473a-886b-760191df069f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:50.343083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c59756-3058-46f8-beae-42021f446d17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:50.344407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c59756-3058-46f8-beae-42021f446d17 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:50.344696 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c59756-3058-46f8-beae-42021f446d17 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:50.345949 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c59756-3058-46f8-beae-42021f446d17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:50.370358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c59756-3058-46f8-beae-42021f446d17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:50.374820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c59756-3058-46f8-beae-42021f446d17 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:50.376790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296d7f-c44d-4526-99d3-f8ceafc034ae None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:50.393782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1263/2512] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:54:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:54:50.403491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19fa10-b125-4de8-9750-792ce0644076 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:50.403491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19fa10-b125-4de8-9750-792ce0644076 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:50.403491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19fa10-b125-4de8-9750-792ce0644076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:50.405988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f6e6e-6205-4e07-a60c-ebb90168abf8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:50.407444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f6e6e-6205-4e07-a60c-ebb90168abf8 service swift] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:50.407866 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f6e6e-6205-4e07-a60c-ebb90168abf8 service swift] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:50.408287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f6e6e-6205-4e07-a60c-ebb90168abf8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:50.438376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f6e6e-6205-4e07-a60c-ebb90168abf8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:50.440123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f6e6e-6205-4e07-a60c-ebb90168abf8 service swift] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:50.469865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1250/2513] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:54:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:54:50.478233 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6a19fa10-b125-4de8-9750-792ce0644076 None None] MFA Rules not processed for user `89faad7b1c26412cac5c2c624fa3f0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:54:50.481632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6a19fa10-b125-4de8-9750-792ce0644076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:50.604400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1264/2514] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:54:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:54:51.308565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f6e6e-6205-4e07-a60c-ebb90168abf8 service swift] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:51.310178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f74f6e6e-6205-4e07-a60c-ebb90168abf8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:51.342826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f6e6e-6205-4e07-a60c-ebb90168abf8 service swift] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:51.350305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2eba8f-8862-40ca-acea-e7f765aaa5f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:51.351967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eba8f-8862-40ca-acea-e7f765aaa5f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:51.354073 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eba8f-8862-40ca-acea-e7f765aaa5f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:51.357089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eba8f-8862-40ca-acea-e7f765aaa5f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:51.359544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6f2eba8f-8862-40ca-acea-e7f765aaa5f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:51.375183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2eba8f-8862-40ca-acea-e7f765aaa5f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:51.377968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2eba8f-8862-40ca-acea-e7f765aaa5f2 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:51.403881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1251/2515] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:54:56.650742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d1b5d-9d12-4156-905c-aa0ebbce2118 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:56.652698 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d1b5d-9d12-4156-905c-aa0ebbce2118 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:56.653196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d1b5d-9d12-4156-905c-aa0ebbce2118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:56.725208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1f2d1b5d-9d12-4156-905c-aa0ebbce2118 None None] MFA Rules not processed for user `3f46252462244a0a84180c4726d15f94`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:54:56.726874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1f2d1b5d-9d12-4156-905c-aa0ebbce2118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:56.908693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1265/2516] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:54:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:54:56.926300 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2eba8f-8862-40ca-acea-e7f765aaa5f2 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:54:56.928187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6f2eba8f-8862-40ca-acea-e7f765aaa5f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:54:57.112141 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2eba8f-8862-40ca-acea-e7f765aaa5f2 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:54:57.122869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b7127-fcbf-43fc-a244-c7a699d28b0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:54:57.123949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b7127-fcbf-43fc-a244-c7a699d28b0b service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:54:57.124395 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b7127-fcbf-43fc-a244-c7a699d28b0b service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:54:57.124827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b7127-fcbf-43fc-a244-c7a699d28b0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:54:57.140033 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b7127-fcbf-43fc-a244-c7a699d28b0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:54:57.142795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b7127-fcbf-43fc-a244-c7a699d28b0b service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:54:57.171335 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1252/2517] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:54:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:55:02.096068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9deba6-44de-4388-bf11-7a4098070dce None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:02.096566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9deba6-44de-4388-bf11-7a4098070dce None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:02.096668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9deba6-44de-4388-bf11-7a4098070dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:02.122939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-af9deba6-44de-4388-bf11-7a4098070dce None None] MFA Rules not processed for user `3f46252462244a0a84180c4726d15f94`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:55:02.127086 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-af9deba6-44de-4388-bf11-7a4098070dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:02.145818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1266/2518] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:55:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:55:07.154333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=82 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:55:07.430760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b7127-fcbf-43fc-a244-c7a699d28b0b service nova] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:55:07.432350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-758b7127-fcbf-43fc-a244-c7a699d28b0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:07.456083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b7127-fcbf-43fc-a244-c7a699d28b0b service nova] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:55:07.463432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:55:07.464919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:07.465376 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:07.465833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] PATH_INFO: `/v3/users/fb627e22492140858b12dcf4c0a70821` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:07.467628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb627e22492140858b12dcf4c0a70821)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:55:07.470443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:55:07.699204 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb627e22492140858b12dcf4c0a70821'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:07.811316 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb627e22492140858b12dcf4c0a70821'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:07.812191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb627e22492140858b12dcf4c0a70821'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:07.825104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb627e22492140858b12dcf4c0a70821'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:07.851560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb627e22492140858b12dcf4c0a70821'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:07.863310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1253/2519] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:55:07 2022] DELETE /identity/v3/users/fb627e22492140858b12dcf4c0a70821 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:55:08.316463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9deba6-44de-4388-bf11-7a4098070dce None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:55:08.329242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9deba6-44de-4388-bf11-7a4098070dce None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:55:08.414970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:55:08.469266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:55:08.469992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:08.470277 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:08.470560 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] PATH_INFO: `/v3/projects/0cdb8140dca34727ab99c95efc7acb81` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:08.471782 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cdb8140dca34727ab99c95efc7acb81)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:55:08.474581 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:55:08.610973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cdb8140dca34727ab99c95efc7acb81', 'request_id': 'req-b8644850-da14-48dc-b7ed-4edfb8f8b134'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:08.621063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cdb8140dca34727ab99c95efc7acb81', 'request_id': 'req-b8644850-da14-48dc-b7ed-4edfb8f8b134'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:08.686371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cdb8140dca34727ab99c95efc7acb81', 'request_id': 'req-b8644850-da14-48dc-b7ed-4edfb8f8b134'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:08.710111 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] The token cache is being invalidate because project 0cdb8140dca34727ab99c95efc7acb81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:55:08.711165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:08.715834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:08.723432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1267/2520] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:55:08 2022] DELETE /identity/v3/projects/0cdb8140dca34727ab99c95efc7acb81 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:55:08.724664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:55:08.730195 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:08.755130 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14906603-5a12-47b6-a601-dac2d0e7eca2 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:55:08.879907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:55:08.880862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:08.881403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:08.881872 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] PATH_INFO: `/v3/users/abc8b9fe160546faada4e40eed48047d` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:08.883238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc8b9fe160546faada4e40eed48047d)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:55:08.885538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:55:09.400090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc8b9fe160546faada4e40eed48047d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:09.572615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc8b9fe160546faada4e40eed48047d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:09.573227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc8b9fe160546faada4e40eed48047d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:09.584222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc8b9fe160546faada4e40eed48047d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:09.592859 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:55:09.597603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-219d7c8b-f052-4fd7-a2ae-4e1e7e2ac2e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc8b9fe160546faada4e40eed48047d'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:09.606635 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1254/2521] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:55:08 2022] DELETE /identity/v3/users/abc8b9fe160546faada4e40eed48047d => generated 0 bytes in 883 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:55:10.045391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:55:10.047662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:10.073452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8644850-da14-48dc-b7ed-4edfb8f8b134 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:55:10.166444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:55:10.167424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:10.168346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:10.168817 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] PATH_INFO: `/v3/projects/958f75cfbe2c41349fa0e91fcd80e2f9` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:10.171280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=958f75cfbe2c41349fa0e91fcd80e2f9)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:55:10.173437 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:55:10.388521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '958f75cfbe2c41349fa0e91fcd80e2f9', 'request_id': 'req-6167158f-77f1-4b0c-b91c-e66b32905a10'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:10.430221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '958f75cfbe2c41349fa0e91fcd80e2f9', 'request_id': 'req-6167158f-77f1-4b0c-b91c-e66b32905a10'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:10.510848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '958f75cfbe2c41349fa0e91fcd80e2f9', 'request_id': 'req-6167158f-77f1-4b0c-b91c-e66b32905a10'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:10.516786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:55:10.528168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] The token cache is being invalidate because project 958f75cfbe2c41349fa0e91fcd80e2f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:55:10.528578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:10.530066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6167158f-77f1-4b0c-b91c-e66b32905a10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:55:10.532780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1268/2522] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:55:10 2022] DELETE /identity/v3/projects/958f75cfbe2c41349fa0e91fcd80e2f9 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:55:20.184713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:55:20.268079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfeaf14-5b70-4e63-8c29-1ae1f34deea7 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:20.268079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfeaf14-5b70-4e63-8c29-1ae1f34deea7 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:20.268377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfeaf14-5b70-4e63-8c29-1ae1f34deea7 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:20.270593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1255/2523] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:55:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:55:20.277671 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b193a-4e6c-42aa-9154-bff15fe6220f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:20.278285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b193a-4e6c-42aa-9154-bff15fe6220f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:20.278285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b193a-4e6c-42aa-9154-bff15fe6220f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:20.313519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-320b193a-4e6c-42aa-9154-bff15fe6220f None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:55:20.315258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-320b193a-4e6c-42aa-9154-bff15fe6220f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:20.437756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1269/2524] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:55:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:55:20.450093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfeaf14-5b70-4e63-8c29-1ae1f34deea7 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:55:20.451880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-cdfeaf14-5b70-4e63-8c29-1ae1f34deea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:20.513124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfeaf14-5b70-4e63-8c29-1ae1f34deea7 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:55:20.516663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff5684-1b43-4b19-bf4d-c2e19380fc04 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab54ee52004746eaadb5c2970e59afc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:55:20.517262 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff5684-1b43-4b19-bf4d-c2e19380fc04 service glance] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:20.517541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff5684-1b43-4b19-bf4d-c2e19380fc04 service glance] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:20.517854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff5684-1b43-4b19-bf4d-c2e19380fc04 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:20.528389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff5684-1b43-4b19-bf4d-c2e19380fc04 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:55:20.529806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff5684-1b43-4b19-bf4d-c2e19380fc04 service glance] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:55:20.541496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1256/2525] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:55:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:55:20.754890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd2958-6ce5-49ed-8430-81dd810c5cbc None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:20.755177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd2958-6ce5-49ed-8430-81dd810c5cbc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:20.755471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd2958-6ce5-49ed-8430-81dd810c5cbc None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:20.756832 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1270/2526] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:55:20.761843 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24729d-8d8d-4ff3-bfc6-f6fedfcab4f2 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:20.762129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24729d-8d8d-4ff3-bfc6-f6fedfcab4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:20.762403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24729d-8d8d-4ff3-bfc6-f6fedfcab4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:20.795336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-2d24729d-8d8d-4ff3-bfc6-f6fedfcab4f2 None None] MFA Rules not processed for user `6336f0c469b84db78a90cf8435c7dbe4`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:55:20.796488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-2d24729d-8d8d-4ff3-bfc6-f6fedfcab4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:20.861903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1257/2527] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:55:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:55:20.877181 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd2958-6ce5-49ed-8430-81dd810c5cbc None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:55:20.878803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d6bd2958-6ce5-49ed-8430-81dd810c5cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:20.891971 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd2958-6ce5-49ed-8430-81dd810c5cbc None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:55:20.895444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd9894-1ce5-41e7-8337-2065dea3849a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:55:20.896031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd9894-1ce5-41e7-8337-2065dea3849a service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:20.896308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd9894-1ce5-41e7-8337-2065dea3849a service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:20.896578 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd9894-1ce5-41e7-8337-2065dea3849a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:20.904174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd9894-1ce5-41e7-8337-2065dea3849a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:55:20.905586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd9894-1ce5-41e7-8337-2065dea3849a service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:55:20.913924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1271/2528] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:55:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:55:21.032446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158d338-d335-4ae1-bf45-2fec475c3238 None None] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:21.033280 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158d338-d335-4ae1-bf45-2fec475c3238 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:21.033555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158d338-d335-4ae1-bf45-2fec475c3238 None None] PATH_INFO: `/` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:21.035016 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1258/2529] 10.222.0.42 () {54 vars in 921 bytes} [Sat Sep 17 03:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 03:55:21.041438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff8ffc-fcbe-4d2b-b310-edce1113c0e4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:21.041745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff8ffc-fcbe-4d2b-b310-edce1113c0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:21.042022 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff8ffc-fcbe-4d2b-b310-edce1113c0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:21.062443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8fff8ffc-fcbe-4d2b-b310-edce1113c0e4 None None] MFA Rules not processed for user `9901608b62344115aa561d00bfe88c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:55:21.063575 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8fff8ffc-fcbe-4d2b-b310-edce1113c0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:21.073585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1272/2530] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Sep 17 03:55:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:55:21.084358 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158d338-d335-4ae1-bf45-2fec475c3238 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:55:21.086651 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a158d338-d335-4ae1-bf45-2fec475c3238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:21.100108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158d338-d335-4ae1-bf45-2fec475c3238 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:55:21.104342 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcf39c-1cbc-46d4-bbfe-3ed12ed30b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:55:21.105008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcf39c-1cbc-46d4-bbfe-3ed12ed30b52 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:21.105381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcf39c-1cbc-46d4-bbfe-3ed12ed30b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:21.105739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcf39c-1cbc-46d4-bbfe-3ed12ed30b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:21.116092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcf39c-1cbc-46d4-bbfe-3ed12ed30b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:55:21.117555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcf39c-1cbc-46d4-bbfe-3ed12ed30b52 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:55:21.128703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1259/2531] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:55:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:55:23.988996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7faf4e-0dfb-4e03-8fd7-b7348e9fc740 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:23.989455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7faf4e-0dfb-4e03-8fd7-b7348e9fc740 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:23.989580 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7faf4e-0dfb-4e03-8fd7-b7348e9fc740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:24.009905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1d7faf4e-0dfb-4e03-8fd7-b7348e9fc740 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:55:24.011116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1d7faf4e-0dfb-4e03-8fd7-b7348e9fc740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:24.065702 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1273/2532] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:55:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:55:24.078486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcf39c-1cbc-46d4-bbfe-3ed12ed30b52 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:55:24.079881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1ffcf39c-1cbc-46d4-bbfe-3ed12ed30b52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:24.149336 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcf39c-1cbc-46d4-bbfe-3ed12ed30b52 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:55:24.154244 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aacb5c-448b-4b76-b693-97097ca33bc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:55:24.154836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aacb5c-448b-4b76-b693-97097ca33bc4 service swift] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:24.155112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aacb5c-448b-4b76-b693-97097ca33bc4 service swift] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:24.155383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aacb5c-448b-4b76-b693-97097ca33bc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:24.163215 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aacb5c-448b-4b76-b693-97097ca33bc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:55:24.164951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aacb5c-448b-4b76-b693-97097ca33bc4 service swift] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:55:24.173275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1260/2533] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:55:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:55:34.059110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:55:34.170078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=96 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:55:42.870395 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346353d-0d96-448e-8969-35f6264a3784 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:42.870879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346353d-0d96-448e-8969-35f6264a3784 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:42.870975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346353d-0d96-448e-8969-35f6264a3784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:42.890723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5346353d-0d96-448e-8969-35f6264a3784 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:55:42.891947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5346353d-0d96-448e-8969-35f6264a3784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:42.899594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1274/2534] 10.222.0.42 () {58 vars in 967 bytes} [Sat Sep 17 03:55:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:55:42.913678 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aacb5c-448b-4b76-b693-97097ca33bc4 service swift] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:55:42.925691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aacb5c-448b-4b76-b693-97097ca33bc4 service swift] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:55:42.930811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dac022-96ee-4f7e-ac34-d85d4094a60a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:55:42.931448 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dac022-96ee-4f7e-ac34-d85d4094a60a service swift] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:42.931719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dac022-96ee-4f7e-ac34-d85d4094a60a service swift] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:42.931996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dac022-96ee-4f7e-ac34-d85d4094a60a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:42.944152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dac022-96ee-4f7e-ac34-d85d4094a60a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:55:42.945930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dac022-96ee-4f7e-ac34-d85d4094a60a service swift] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:55:42.957477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1261/2535] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:55:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:55:52.885136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:55:52.953490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=6 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:55:58.923788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346353d-0d96-448e-8969-35f6264a3784 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:55:58.933112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346353d-0d96-448e-8969-35f6264a3784 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:55:58.937856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f7224-2251-4c42-9994-673b5de7b1a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:55:58.938644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f7224-2251-4c42-9994-673b5de7b1a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:55:58.939103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f7224-2251-4c42-9994-673b5de7b1a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:55:58.939533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f7224-2251-4c42-9994-673b5de7b1a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:55:58.941563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-081f7224-2251-4c42-9994-673b5de7b1a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:55:59.027547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f7224-2251-4c42-9994-673b5de7b1a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:55:59.029385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f7224-2251-4c42-9994-673b5de7b1a5 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:55:59.048619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1275/2536] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:02.449863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dac022-96ee-4f7e-ac34-d85d4094a60a service swift] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:02.451417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-01dac022-96ee-4f7e-ac34-d85d4094a60a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:02.531530 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dac022-96ee-4f7e-ac34-d85d4094a60a service swift] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:02.536545 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b4be5-25d3-4061-9e9a-d2260e5947a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ec5c9c71d14762a4869b7d66e87ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:02.537155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b4be5-25d3-4061-9e9a-d2260e5947a6 service placement] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:02.537440 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b4be5-25d3-4061-9e9a-d2260e5947a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:02.537716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b4be5-25d3-4061-9e9a-d2260e5947a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:02.539550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bc3b4be5-25d3-4061-9e9a-d2260e5947a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:02.552241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b4be5-25d3-4061-9e9a-d2260e5947a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:02.553864 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b4be5-25d3-4061-9e9a-d2260e5947a6 service placement] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:02.564730 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1262/2537] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:02.596720 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f7224-2251-4c42-9994-673b5de7b1a5 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:02.597864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-081f7224-2251-4c42-9994-673b5de7b1a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:02.611177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f7224-2251-4c42-9994-673b5de7b1a5 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:02.616068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:02.616692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:02.616973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:02.617248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] PATH_INFO: `/v3/users/fafa609d94334db6907b401e66cb3448` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:02.618772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafa609d94334db6907b401e66cb3448)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:02.620293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:02.814921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafa609d94334db6907b401e66cb3448'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:02.827147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafa609d94334db6907b401e66cb3448'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:02.908149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafa609d94334db6907b401e66cb3448'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:02.908795 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafa609d94334db6907b401e66cb3448'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:02.916573 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-29df986b-2714-4f96-b6ca-d062203907e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafa609d94334db6907b401e66cb3448'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:02.923895 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1276/2538] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:02 2022] DELETE /identity/v3/users/fafa609d94334db6907b401e66cb3448 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:03.132974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b4be5-25d3-4061-9e9a-d2260e5947a6 service placement] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:03.144205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b4be5-25d3-4061-9e9a-d2260e5947a6 service placement] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:03.206381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:03.207005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:03.207285 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:03.207558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] PATH_INFO: `/v3/projects/fb6cb03abf5242b594efcf325909dd32` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:03.208683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6cb03abf5242b594efcf325909dd32)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:03.210651 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:03.301526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6cb03abf5242b594efcf325909dd32', 'request_id': 'req-225526a0-45cb-4ff9-b74a-699b91fc64dc'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:03.364702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6cb03abf5242b594efcf325909dd32', 'request_id': 'req-225526a0-45cb-4ff9-b74a-699b91fc64dc'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:03.370499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6cb03abf5242b594efcf325909dd32', 'request_id': 'req-225526a0-45cb-4ff9-b74a-699b91fc64dc'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:03.382458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] The token cache is being invalidate because project fb6cb03abf5242b594efcf325909dd32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:56:03.382675 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:03.383301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-225526a0-45cb-4ff9-b74a-699b91fc64dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:03.384768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1263/2539] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:56:03 2022] DELETE /identity/v3/projects/fb6cb03abf5242b594efcf325909dd32 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:03.389878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce655bb8-3fa2-4d15-8aee-474397bf30d2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:03.390171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce655bb8-3fa2-4d15-8aee-474397bf30d2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:03.390450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce655bb8-3fa2-4d15-8aee-474397bf30d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:03.428065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ce655bb8-3fa2-4d15-8aee-474397bf30d2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:03.429285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ce655bb8-3fa2-4d15-8aee-474397bf30d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:03.456672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:03.490062 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1277/2540] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:03.498784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b83dd4-24ca-4e14-bb64-cbeec847cd7e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:03.499266 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b83dd4-24ca-4e14-bb64-cbeec847cd7e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:03.499478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b83dd4-24ca-4e14-bb64-cbeec847cd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:03.531519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-49b83dd4-24ca-4e14-bb64-cbeec847cd7e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:03.533404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-49b83dd4-24ca-4e14-bb64-cbeec847cd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:03.543321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1264/2541] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:03.548409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce655bb8-3fa2-4d15-8aee-474397bf30d2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:03.559454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce655bb8-3fa2-4d15-8aee-474397bf30d2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:03.564215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c8cc9-7f85-4e8f-8de8-d9a43904c8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:03.566693 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c8cc9-7f85-4e8f-8de8-d9a43904c8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:03.566976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c8cc9-7f85-4e8f-8de8-d9a43904c8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:03.567251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c8cc9-7f85-4e8f-8de8-d9a43904c8a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:03.569120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c8cc9-7f85-4e8f-8de8-d9a43904c8a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:56:03.569426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c8cc9-7f85-4e8f-8de8-d9a43904c8a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:03.572170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c8cc9-7f85-4e8f-8de8-d9a43904c8a8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:03.592851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1278/2542] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:56:03.597964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b83dd4-24ca-4e14-bb64-cbeec847cd7e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:03.610124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b83dd4-24ca-4e14-bb64-cbeec847cd7e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:03.614906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4332f-a9d7-4d06-beb2-d0b7bd0ccfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:03.615513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4332f-a9d7-4d06-beb2-d0b7bd0ccfba admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:03.615791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4332f-a9d7-4d06-beb2-d0b7bd0ccfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:03.616063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4332f-a9d7-4d06-beb2-d0b7bd0ccfba admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:03.616063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4332f-a9d7-4d06-beb2-d0b7bd0ccfba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:03.619190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4332f-a9d7-4d06-beb2-d0b7bd0ccfba admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:03.632546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:03.719424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1265/2543] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:56:03 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:03.723734 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c8cc9-7f85-4e8f-8de8-d9a43904c8a8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:03.738122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c8cc9-7f85-4e8f-8de8-d9a43904c8a8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:03.815518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0782a-f79a-48e2-b19d-c63e1e198a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:03.816130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0782a-f79a-48e2-b19d-c63e1e198a71 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:03.816409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0782a-f79a-48e2-b19d-c63e1e198a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:03.816774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0782a-f79a-48e2-b19d-c63e1e198a71 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:03.817410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0782a-f79a-48e2-b19d-c63e1e198a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:03.819490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0782a-f79a-48e2-b19d-c63e1e198a71 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:03.892013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1279/2544] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:56:03 2022] POST /identity/v3/users => generated 303 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:03.896119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4332f-a9d7-4d06-beb2-d0b7bd0ccfba admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:03.911286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4332f-a9d7-4d06-beb2-d0b7bd0ccfba admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:03.918769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f2f03-1d08-4d9b-aae9-6e038e2960c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:03.919590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f2f03-1d08-4d9b-aae9-6e038e2960c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:03.919873 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f2f03-1d08-4d9b-aae9-6e038e2960c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:03.920182 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f2f03-1d08-4d9b-aae9-6e038e2960c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:03.920840 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f2f03-1d08-4d9b-aae9-6e038e2960c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:03.922487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f2f03-1d08-4d9b-aae9-6e038e2960c6 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:03.933410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1266/2545] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:56:03.938137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0782a-f79a-48e2-b19d-c63e1e198a71 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:03.949083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0782a-f79a-48e2-b19d-c63e1e198a71 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:03.954630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9d38d-b17b-47bf-bf0a-e006e9bd55fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:03.955245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9d38d-b17b-47bf-bf0a-e006e9bd55fd admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:03.955516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9d38d-b17b-47bf-bf0a-e006e9bd55fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:03.955790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9d38d-b17b-47bf-bf0a-e006e9bd55fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:03.956432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9d38d-b17b-47bf-bf0a-e006e9bd55fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:03.958108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9d38d-b17b-47bf-bf0a-e006e9bd55fd admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:03.971030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1280/2546] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:56:03.976582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f2f03-1d08-4d9b-aae9-6e038e2960c6 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:03.988834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f2f03-1d08-4d9b-aae9-6e038e2960c6 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:03.993985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5711b-efb4-467b-940b-2854966e0703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:03.994643 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5711b-efb4-467b-940b-2854966e0703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:03.994919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5711b-efb4-467b-940b-2854966e0703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:03.995191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5711b-efb4-467b-940b-2854966e0703 admin admin] PATH_INFO: `/v3/projects/66aefb9d95864b59ad36a0047963f83e/users/f4328efff0624bf598181c57c44d66bc/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:04.011683 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5711b-efb4-467b-940b-2854966e0703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66aefb9d95864b59ad36a0047963f83e, user_id=f4328efff0624bf598181c57c44d66bc, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:04.015034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5711b-efb4-467b-940b-2854966e0703 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:04.077452 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1267/2547] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:03 2022] PUT /identity/v3/projects/66aefb9d95864b59ad36a0047963f83e/users/f4328efff0624bf598181c57c44d66bc/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:04.095736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2fee-4a66-4832-a87f-c1028c1b702d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:04.095736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2fee-4a66-4832-a87f-c1028c1b702d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:04.095736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2fee-4a66-4832-a87f-c1028c1b702d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:04.120640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-393b2fee-4a66-4832-a87f-c1028c1b702d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:04.122400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-393b2fee-4a66-4832-a87f-c1028c1b702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:04.206438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1281/2548] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:04.217718 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5711b-efb4-467b-940b-2854966e0703 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:04.220288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c6d5711b-efb4-467b-940b-2854966e0703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:04.284917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5711b-efb4-467b-940b-2854966e0703 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:04.288673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef545c-bb3c-433a-aaee-3d86bd65c2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:04.289274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef545c-bb3c-433a-aaee-3d86bd65c2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:04.289578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef545c-bb3c-433a-aaee-3d86bd65c2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:04.289884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef545c-bb3c-433a-aaee-3d86bd65c2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:04.301915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef545c-bb3c-433a-aaee-3d86bd65c2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:04.303377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef545c-bb3c-433a-aaee-3d86bd65c2b3 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:04.314155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1268/2549] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:05.669475 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57c72e-fe1e-4bae-ada6-c44cbc380074 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:05.670141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57c72e-fe1e-4bae-ada6-c44cbc380074 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:05.670141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57c72e-fe1e-4bae-ada6-c44cbc380074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:05.700186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5f57c72e-fe1e-4bae-ada6-c44cbc380074 None None] MFA Rules not processed for user `f4328efff0624bf598181c57c44d66bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:05.701440 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5f57c72e-fe1e-4bae-ada6-c44cbc380074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:05.728391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:05.768568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1282/2550] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:05 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:05.772917 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef545c-bb3c-433a-aaee-3d86bd65c2b3 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:05.783357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef545c-bb3c-433a-aaee-3d86bd65c2b3 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:05.789069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e532761-84c6-4f14-8285-44564ede8433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:05.789655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532761-84c6-4f14-8285-44564ede8433 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:05.789976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532761-84c6-4f14-8285-44564ede8433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:05.790245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532761-84c6-4f14-8285-44564ede8433 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:05.790876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e532761-84c6-4f14-8285-44564ede8433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:05.792838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e532761-84c6-4f14-8285-44564ede8433 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:05.938142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1269/2551] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:56:05 2022] POST /identity/v3/projects => generated 405 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:05.942254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57c72e-fe1e-4bae-ada6-c44cbc380074 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:05.953361 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57c72e-fe1e-4bae-ada6-c44cbc380074 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:06.011816 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7132-9f78-45b7-a8e4-b35db36933e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:06.012736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7132-9f78-45b7-a8e4-b35db36933e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:06.013014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7132-9f78-45b7-a8e4-b35db36933e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:06.013289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7132-9f78-45b7-a8e4-b35db36933e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:06.013990 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa7132-9f78-45b7-a8e4-b35db36933e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:06.015996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa7132-9f78-45b7-a8e4-b35db36933e5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:06.105590 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1283/2552] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:56:05 2022] POST /identity/v3/users => generated 309 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:06.107912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e532761-84c6-4f14-8285-44564ede8433 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:06.118383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e532761-84c6-4f14-8285-44564ede8433 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:06.123276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bb32e-515f-4306-ad5e-af59e1139cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:06.124017 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb32e-515f-4306-ad5e-af59e1139cec admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:06.124233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb32e-515f-4306-ad5e-af59e1139cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:06.124513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb32e-515f-4306-ad5e-af59e1139cec admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:06.125206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bb32e-515f-4306-ad5e-af59e1139cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:06.136233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bb32e-515f-4306-ad5e-af59e1139cec admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:06.138717 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1270/2553] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:56:06.143001 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7132-9f78-45b7-a8e4-b35db36933e5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:06.153819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7132-9f78-45b7-a8e4-b35db36933e5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:06.158654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b23b6a-2979-4c3d-84bc-87f770aaf320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:06.160481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23b6a-2979-4c3d-84bc-87f770aaf320 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:06.160481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23b6a-2979-4c3d-84bc-87f770aaf320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:06.160481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23b6a-2979-4c3d-84bc-87f770aaf320 admin admin] PATH_INFO: `/v3/projects/b3c6f83e8b1b4f36ae469348a2395273/users/21348285d0d64c49b2c26cd25a2e0653/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:06.175601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b23b6a-2979-4c3d-84bc-87f770aaf320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c6f83e8b1b4f36ae469348a2395273, user_id=21348285d0d64c49b2c26cd25a2e0653, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:06.178397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b23b6a-2979-4c3d-84bc-87f770aaf320 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:06.230035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1284/2554] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:06 2022] PUT /identity/v3/projects/b3c6f83e8b1b4f36ae469348a2395273/users/21348285d0d64c49b2c26cd25a2e0653/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:06.234795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bb32e-515f-4306-ad5e-af59e1139cec admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:06.244977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bb32e-515f-4306-ad5e-af59e1139cec admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:06.308825 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a223c-bf35-4170-9241-790ad27bd4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:06.310155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a223c-bf35-4170-9241-790ad27bd4ce admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:06.310441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a223c-bf35-4170-9241-790ad27bd4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:06.310713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a223c-bf35-4170-9241-790ad27bd4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:06.311354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a223c-bf35-4170-9241-790ad27bd4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:06.312954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a223c-bf35-4170-9241-790ad27bd4ce admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:06.322746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1271/2555] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:06.327026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b23b6a-2979-4c3d-84bc-87f770aaf320 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:06.337338 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b23b6a-2979-4c3d-84bc-87f770aaf320 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:06.341965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfa770-cb10-4368-8049-9339ceee2de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:06.342606 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfa770-cb10-4368-8049-9339ceee2de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:06.342884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfa770-cb10-4368-8049-9339ceee2de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:06.343158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfa770-cb10-4368-8049-9339ceee2de5 admin admin] PATH_INFO: `/v3/projects/b3c6f83e8b1b4f36ae469348a2395273/users/21348285d0d64c49b2c26cd25a2e0653/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:06.345357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfa770-cb10-4368-8049-9339ceee2de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c6f83e8b1b4f36ae469348a2395273, user_id=21348285d0d64c49b2c26cd25a2e0653, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:06.348070 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfa770-cb10-4368-8049-9339ceee2de5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:06.365480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1285/2556] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:06 2022] PUT /identity/v3/projects/b3c6f83e8b1b4f36ae469348a2395273/users/21348285d0d64c49b2c26cd25a2e0653/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:07.509806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c7888-c0f4-4759-a55e-2ba4d015927d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:07.510305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c7888-c0f4-4759-a55e-2ba4d015927d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:07.510560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c7888-c0f4-4759-a55e-2ba4d015927d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:07.544768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-824c7888-c0f4-4759-a55e-2ba4d015927d None None] MFA Rules not processed for user `21348285d0d64c49b2c26cd25a2e0653`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:07.544768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-824c7888-c0f4-4759-a55e-2ba4d015927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:07.555820 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:07.621245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1272/2557] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:07 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:08.988420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfa770-cb10-4368-8049-9339ceee2de5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:09.055276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfa770-cb10-4368-8049-9339ceee2de5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:09.060042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:09.060675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:09.060957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:09.061236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] PATH_INFO: `/v3/users/f4328efff0624bf598181c57c44d66bc` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:09.062391 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4328efff0624bf598181c57c44d66bc)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:09.064310 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:09.087145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:09.247243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4328efff0624bf598181c57c44d66bc'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:09.260609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4328efff0624bf598181c57c44d66bc'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:09.319567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4328efff0624bf598181c57c44d66bc'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:09.320126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4328efff0624bf598181c57c44d66bc'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:09.328433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4328efff0624bf598181c57c44d66bc'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:09.334935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1286/2558] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:08 2022] DELETE /identity/v3/users/f4328efff0624bf598181c57c44d66bc => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:09.578837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c7888-c0f4-4759-a55e-2ba4d015927d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:09.590839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c7888-c0f4-4759-a55e-2ba4d015927d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:09.650175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:09.650721 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:09.651002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:09.651276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] PATH_INFO: `/v3/projects/66aefb9d95864b59ad36a0047963f83e` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:09.652386 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66aefb9d95864b59ad36a0047963f83e)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:09.654337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:09.760174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66aefb9d95864b59ad36a0047963f83e', 'request_id': 'req-728e667d-a01c-49c8-bfe2-ab6372003c50'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:09.828804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66aefb9d95864b59ad36a0047963f83e', 'request_id': 'req-728e667d-a01c-49c8-bfe2-ab6372003c50'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:09.834884 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66aefb9d95864b59ad36a0047963f83e', 'request_id': 'req-728e667d-a01c-49c8-bfe2-ab6372003c50'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:09.847207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] The token cache is being invalidate because project 66aefb9d95864b59ad36a0047963f83e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:56:09.847494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:09.848129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:09.849696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1273/2559] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:56:09 2022] DELETE /identity/v3/projects/66aefb9d95864b59ad36a0047963f83e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:09.853792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:09.855224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:09.869099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdd609-11b4-409b-b3b6-a27be7d8db60 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:09.925099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:09.925741 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:09.926029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:09.926304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] PATH_INFO: `/v3/users/21348285d0d64c49b2c26cd25a2e0653` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:09.927432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21348285d0d64c49b2c26cd25a2e0653)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:09.929345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:10.127075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21348285d0d64c49b2c26cd25a2e0653'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:10.138781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21348285d0d64c49b2c26cd25a2e0653'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:10.228069 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21348285d0d64c49b2c26cd25a2e0653'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:10.228714 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21348285d0d64c49b2c26cd25a2e0653'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:10.237209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-407847fb-acaa-40b8-93da-5714aed98158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21348285d0d64c49b2c26cd25a2e0653'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:10.243218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1287/2560] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:09 2022] DELETE /identity/v3/users/21348285d0d64c49b2c26cd25a2e0653 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:10.519670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:10.521069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:10.535534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e667d-a01c-49c8-bfe2-ab6372003c50 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:10.583619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:10.608213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:10.608838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:10.609222 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:10.609549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] PATH_INFO: `/v3/projects/b3c6f83e8b1b4f36ae469348a2395273` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:10.610705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c6f83e8b1b4f36ae469348a2395273)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:10.612615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:10.715363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c6f83e8b1b4f36ae469348a2395273', 'request_id': 'req-21508ddd-b766-4103-90cc-644bfcf8c5aa'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:10.794180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c6f83e8b1b4f36ae469348a2395273', 'request_id': 'req-21508ddd-b766-4103-90cc-644bfcf8c5aa'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:10.800631 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c6f83e8b1b4f36ae469348a2395273', 'request_id': 'req-21508ddd-b766-4103-90cc-644bfcf8c5aa'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:10.812457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] The token cache is being invalidate because project b3c6f83e8b1b4f36ae469348a2395273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:56:10.817526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:10.817526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-21508ddd-b766-4103-90cc-644bfcf8c5aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:10.817526 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1274/2561] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:56:10 2022] DELETE /identity/v3/projects/b3c6f83e8b1b4f36ae469348a2395273 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:10.820014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5bdf-a071-41b9-807f-496d0f319c0e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:10.820299 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5bdf-a071-41b9-807f-496d0f319c0e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:10.820678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5bdf-a071-41b9-807f-496d0f319c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:10.840884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-22eb5bdf-a071-41b9-807f-496d0f319c0e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:10.842700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-22eb5bdf-a071-41b9-807f-496d0f319c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:10.854788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:10.902148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1288/2562] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:56:10.910727 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16712876-3693-4e8c-9463-b35a0b26a30d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:10.911012 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16712876-3693-4e8c-9463-b35a0b26a30d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:10.911288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16712876-3693-4e8c-9463-b35a0b26a30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:10.931013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-16712876-3693-4e8c-9463-b35a0b26a30d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:10.932216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-16712876-3693-4e8c-9463-b35a0b26a30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:10.940838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1275/2563] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:10.945029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb5bdf-a071-41b9-807f-496d0f319c0e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:10.955526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb5bdf-a071-41b9-807f-496d0f319c0e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:10.960133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f2d23-d993-46ba-8486-dad33190dae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:10.960731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f2d23-d993-46ba-8486-dad33190dae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:10.961021 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f2d23-d993-46ba-8486-dad33190dae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:10.961297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f2d23-d993-46ba-8486-dad33190dae0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:10.961985 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f2d23-d993-46ba-8486-dad33190dae0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:56:10.962294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f2d23-d993-46ba-8486-dad33190dae0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:10.963903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f2d23-d993-46ba-8486-dad33190dae0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:10.975864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1289/2564] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:56:10.980247 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16712876-3693-4e8c-9463-b35a0b26a30d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:10.989603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16712876-3693-4e8c-9463-b35a0b26a30d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:10.994290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64456c54-3fa9-42db-bec1-3f5d445ca1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:10.997418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64456c54-3fa9-42db-bec1-3f5d445ca1a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:10.997418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64456c54-3fa9-42db-bec1-3f5d445ca1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:10.997418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64456c54-3fa9-42db-bec1-3f5d445ca1a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:10.997418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64456c54-3fa9-42db-bec1-3f5d445ca1a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:10.998069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64456c54-3fa9-42db-bec1-3f5d445ca1a2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:11.074661 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1276/2565] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:56:10 2022] POST /identity/v3/projects => generated 393 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:11.079694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f2d23-d993-46ba-8486-dad33190dae0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:11.089829 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f2d23-d993-46ba-8486-dad33190dae0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:11.144928 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fa9ed-9feb-4908-b0dd-d8b2e2c38b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:11.145576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fa9ed-9feb-4908-b0dd-d8b2e2c38b91 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:11.145813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fa9ed-9feb-4908-b0dd-d8b2e2c38b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:11.146085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fa9ed-9feb-4908-b0dd-d8b2e2c38b91 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:11.146708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fa9ed-9feb-4908-b0dd-d8b2e2c38b91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:11.148596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fa9ed-9feb-4908-b0dd-d8b2e2c38b91 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:11.227104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1290/2566] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:56:11 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:11.231243 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64456c54-3fa9-42db-bec1-3f5d445ca1a2 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:11.241556 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64456c54-3fa9-42db-bec1-3f5d445ca1a2 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:11.246172 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99098828-62a1-4a51-a9bc-1c43591c3786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:11.246758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99098828-62a1-4a51-a9bc-1c43591c3786 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:11.247032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99098828-62a1-4a51-a9bc-1c43591c3786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:11.247303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99098828-62a1-4a51-a9bc-1c43591c3786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:11.247948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99098828-62a1-4a51-a9bc-1c43591c3786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:11.250081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99098828-62a1-4a51-a9bc-1c43591c3786 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:11.259951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1277/2567] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:11.264484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fa9ed-9feb-4908-b0dd-d8b2e2c38b91 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:11.274483 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fa9ed-9feb-4908-b0dd-d8b2e2c38b91 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:11.278994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f77dc-285f-4517-b280-79ea70e19bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:11.279735 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f77dc-285f-4517-b280-79ea70e19bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:11.280131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f77dc-285f-4517-b280-79ea70e19bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:11.280419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f77dc-285f-4517-b280-79ea70e19bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:11.281066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f77dc-285f-4517-b280-79ea70e19bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:11.282669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f77dc-285f-4517-b280-79ea70e19bd6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:11.293041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1291/2568] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:11.297366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99098828-62a1-4a51-a9bc-1c43591c3786 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:11.307361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99098828-62a1-4a51-a9bc-1c43591c3786 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:11.312903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17c416-f179-4ada-a0c0-c12d44acfbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:11.313539 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17c416-f179-4ada-a0c0-c12d44acfbb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:11.313852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17c416-f179-4ada-a0c0-c12d44acfbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:11.314125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17c416-f179-4ada-a0c0-c12d44acfbb9 admin admin] PATH_INFO: `/v3/projects/d65c1b455ead4d4cbe898ae6c1e8c11a/users/087a49e0795e4c06aaf839daded5d80e/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:11.329036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17c416-f179-4ada-a0c0-c12d44acfbb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65c1b455ead4d4cbe898ae6c1e8c11a, user_id=087a49e0795e4c06aaf839daded5d80e, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:11.331735 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17c416-f179-4ada-a0c0-c12d44acfbb9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:11.399720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1278/2569] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:11 2022] PUT /identity/v3/projects/d65c1b455ead4d4cbe898ae6c1e8c11a/users/087a49e0795e4c06aaf839daded5d80e/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:11.411598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc58716-7c66-44c2-8887-bb2c2127ed9c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:11.411880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc58716-7c66-44c2-8887-bb2c2127ed9c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:11.412151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc58716-7c66-44c2-8887-bb2c2127ed9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:11.441029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3bc58716-7c66-44c2-8887-bb2c2127ed9c None None] MFA Rules not processed for user `087a49e0795e4c06aaf839daded5d80e`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:11.442259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3bc58716-7c66-44c2-8887-bb2c2127ed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:11.507309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1292/2570] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:11.512184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17c416-f179-4ada-a0c0-c12d44acfbb9 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:11.576797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17c416-f179-4ada-a0c0-c12d44acfbb9 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:11.581510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a5429-8203-4bce-b9d8-78fe2dc9cc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:11.582139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a5429-8203-4bce-b9d8-78fe2dc9cc7e admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:11.582427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a5429-8203-4bce-b9d8-78fe2dc9cc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:11.582702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a5429-8203-4bce-b9d8-78fe2dc9cc7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:11.583324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a5429-8203-4bce-b9d8-78fe2dc9cc7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:11.585236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a5429-8203-4bce-b9d8-78fe2dc9cc7e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:11.594976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:11.669338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1279/2571] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:56:11 2022] POST /identity/v3/projects => generated 393 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:11.674142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc58716-7c66-44c2-8887-bb2c2127ed9c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:11.684755 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc58716-7c66-44c2-8887-bb2c2127ed9c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:11.694224 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:11.742597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cad226-d196-4878-b1b6-5aea86bafb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:11.743542 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cad226-d196-4878-b1b6-5aea86bafb4f admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:11.743893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cad226-d196-4878-b1b6-5aea86bafb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:11.744168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cad226-d196-4878-b1b6-5aea86bafb4f admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:11.744794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cad226-d196-4878-b1b6-5aea86bafb4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:11.746787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cad226-d196-4878-b1b6-5aea86bafb4f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:11.851551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1293/2572] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:56:11 2022] POST /identity/v3/users => generated 303 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:11.856727 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a5429-8203-4bce-b9d8-78fe2dc9cc7e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:11.867905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a5429-8203-4bce-b9d8-78fe2dc9cc7e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:11.872585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d3b7c-d30d-43fe-a54d-b9640438142f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:11.873193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d3b7c-d30d-43fe-a54d-b9640438142f admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:11.873488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d3b7c-d30d-43fe-a54d-b9640438142f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:11.873750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d3b7c-d30d-43fe-a54d-b9640438142f admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:11.874424 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d3b7c-d30d-43fe-a54d-b9640438142f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:11.876039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d3b7c-d30d-43fe-a54d-b9640438142f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:11.886345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1280/2573] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:11.890394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cad226-d196-4878-b1b6-5aea86bafb4f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:11.900362 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cad226-d196-4878-b1b6-5aea86bafb4f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:11.905808 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb591aa1-1ea3-419f-b924-78ad874b44fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:11.906471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb591aa1-1ea3-419f-b924-78ad874b44fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:11.906738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb591aa1-1ea3-419f-b924-78ad874b44fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:11.907015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb591aa1-1ea3-419f-b924-78ad874b44fe admin admin] PATH_INFO: `/v3/projects/cc88772a0c534ab887caeab84baeb5fa/users/5ddc3c84923644a79d8f536d504fcef1/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:11.922252 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb591aa1-1ea3-419f-b924-78ad874b44fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc88772a0c534ab887caeab84baeb5fa, user_id=5ddc3c84923644a79d8f536d504fcef1, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:11.924906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb591aa1-1ea3-419f-b924-78ad874b44fe admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:12.004313 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1294/2574] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:11 2022] PUT /identity/v3/projects/cc88772a0c534ab887caeab84baeb5fa/users/5ddc3c84923644a79d8f536d504fcef1/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:12.008273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d3b7c-d30d-43fe-a54d-b9640438142f admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:12.019097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d3b7c-d30d-43fe-a54d-b9640438142f admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:12.075174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0cc48-b6ef-4c92-ae01-f60e81b0660f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:12.075788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0cc48-b6ef-4c92-ae01-f60e81b0660f admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:12.076070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0cc48-b6ef-4c92-ae01-f60e81b0660f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:12.076932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0cc48-b6ef-4c92-ae01-f60e81b0660f admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:12.077710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb0cc48-b6ef-4c92-ae01-f60e81b0660f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:12.079407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb0cc48-b6ef-4c92-ae01-f60e81b0660f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:12.089174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1281/2575] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:12.094164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb591aa1-1ea3-419f-b924-78ad874b44fe admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:12.104819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb591aa1-1ea3-419f-b924-78ad874b44fe admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:12.109887 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457b568-9d56-4625-b529-c97ac6064833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:12.110586 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b568-9d56-4625-b529-c97ac6064833 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:12.110865 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b568-9d56-4625-b529-c97ac6064833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:12.111143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b568-9d56-4625-b529-c97ac6064833 admin admin] PATH_INFO: `/v3/projects/cc88772a0c534ab887caeab84baeb5fa/users/5ddc3c84923644a79d8f536d504fcef1/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:12.113092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457b568-9d56-4625-b529-c97ac6064833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc88772a0c534ab887caeab84baeb5fa, user_id=5ddc3c84923644a79d8f536d504fcef1, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:12.115752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457b568-9d56-4625-b529-c97ac6064833 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:12.126149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1295/2576] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:12 2022] PUT /identity/v3/projects/cc88772a0c534ab887caeab84baeb5fa/users/5ddc3c84923644a79d8f536d504fcef1/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:12.136126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397721d-9abe-4ff4-b8b7-0e466705d6b5 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:12.136415 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397721d-9abe-4ff4-b8b7-0e466705d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:12.136692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397721d-9abe-4ff4-b8b7-0e466705d6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:12.164878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-2397721d-9abe-4ff4-b8b7-0e466705d6b5 None None] MFA Rules not processed for user `5ddc3c84923644a79d8f536d504fcef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:12.166106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-2397721d-9abe-4ff4-b8b7-0e466705d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:12.233484 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1282/2577] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:12 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:56:12.244286 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457b568-9d56-4625-b529-c97ac6064833 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:12.246052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6457b568-9d56-4625-b529-c97ac6064833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:12.324695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457b568-9d56-4625-b529-c97ac6064833 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:12.329046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0c4e2-53bd-43fc-9bfe-43585cf1a92a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:12.329650 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0c4e2-53bd-43fc-9bfe-43585cf1a92a service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:12.329959 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0c4e2-53bd-43fc-9bfe-43585cf1a92a service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:12.330235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0c4e2-53bd-43fc-9bfe-43585cf1a92a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:12.339291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0c4e2-53bd-43fc-9bfe-43585cf1a92a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:12.340721 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0c4e2-53bd-43fc-9bfe-43585cf1a92a service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:12.351516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1296/2578] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:12 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:12.398092 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397721d-9abe-4ff4-b8b7-0e466705d6b5 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:12.399563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-2397721d-9abe-4ff4-b8b7-0e466705d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:12.417273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397721d-9abe-4ff4-b8b7-0e466705d6b5 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:12.425058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fca830-f1ac-4504-b2c3-e2c59e9f480a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:12.425702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fca830-f1ac-4504-b2c3-e2c59e9f480a service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:12.426013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fca830-f1ac-4504-b2c3-e2c59e9f480a service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:12.426290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fca830-f1ac-4504-b2c3-e2c59e9f480a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:12.486383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fca830-f1ac-4504-b2c3-e2c59e9f480a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:12.487702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fca830-f1ac-4504-b2c3-e2c59e9f480a service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:12.491216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:12.497050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1283/2579] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:17.154893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0c4e2-53bd-43fc-9bfe-43585cf1a92a service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:17.204177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:17.217564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0c4e2-53bd-43fc-9bfe-43585cf1a92a service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:17.222995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:17.223614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:17.223894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:17.224177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] PATH_INFO: `/v3/users/087a49e0795e4c06aaf839daded5d80e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:17.225270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=087a49e0795e4c06aaf839daded5d80e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:17.227213 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:17.408240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '087a49e0795e4c06aaf839daded5d80e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:17.420034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '087a49e0795e4c06aaf839daded5d80e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:17.479460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '087a49e0795e4c06aaf839daded5d80e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:17.480096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '087a49e0795e4c06aaf839daded5d80e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:17.488644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '087a49e0795e4c06aaf839daded5d80e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:17.496510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1297/2580] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:17 2022] DELETE /identity/v3/users/087a49e0795e4c06aaf839daded5d80e => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:17.502319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d6fab-1c71-4950-9c6f-28384fb0b43d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:17.502832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d6fab-1c71-4950-9c6f-28384fb0b43d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:17.502928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d6fab-1c71-4950-9c6f-28384fb0b43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:17.523078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ee2d6fab-1c71-4950-9c6f-28384fb0b43d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:17.524286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ee2d6fab-1c71-4950-9c6f-28384fb0b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:17.583823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1284/2581] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:56:17.594308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:17.595991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:17.656784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d6f51-4b17-4409-93f6-4669a5413a83 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:17.660996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa1c39-2cae-436c-a681-07f2ac189b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:17.661550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa1c39-2cae-436c-a681-07f2ac189b0a service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:17.661857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa1c39-2cae-436c-a681-07f2ac189b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:17.662134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa1c39-2cae-436c-a681-07f2ac189b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:17.672362 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa1c39-2cae-436c-a681-07f2ac189b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:17.673841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa1c39-2cae-436c-a681-07f2ac189b0a service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:17.683685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1298/2582] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:18.086693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d6fab-1c71-4950-9c6f-28384fb0b43d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:18.088200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ee2d6fab-1c71-4950-9c6f-28384fb0b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:18.102142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d6fab-1c71-4950-9c6f-28384fb0b43d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:18.106970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:18.107608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:18.107892 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:18.108167 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] PATH_INFO: `/v3/projects/d65c1b455ead4d4cbe898ae6c1e8c11a` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:18.109616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d65c1b455ead4d4cbe898ae6c1e8c11a)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:18.111606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:18.223413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd65c1b455ead4d4cbe898ae6c1e8c11a', 'request_id': 'req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:18.297145 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd65c1b455ead4d4cbe898ae6c1e8c11a', 'request_id': 'req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:18.302977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd65c1b455ead4d4cbe898ae6c1e8c11a', 'request_id': 'req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:18.314666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] The token cache is being invalidate because project d65c1b455ead4d4cbe898ae6c1e8c11a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:56:18.314970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:18.315590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:18.317702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1285/2583] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:56:18 2022] DELETE /identity/v3/projects/d65c1b455ead4d4cbe898ae6c1e8c11a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:18.321410 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa1c39-2cae-436c-a681-07f2ac189b0a service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:18.331547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa1c39-2cae-436c-a681-07f2ac189b0a service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:18.388629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:18.389302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:18.390048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:18.390332 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] PATH_INFO: `/v3/users/5ddc3c84923644a79d8f536d504fcef1` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:18.391455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddc3c84923644a79d8f536d504fcef1)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:18.393372 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:18.405570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:18.579601 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddc3c84923644a79d8f536d504fcef1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:18.591381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddc3c84923644a79d8f536d504fcef1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:18.639149 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddc3c84923644a79d8f536d504fcef1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:18.640191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddc3c84923644a79d8f536d504fcef1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:18.647422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5044714a-39fc-4656-8ec3-0ab98cedf3ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddc3c84923644a79d8f536d504fcef1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:18.653534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1299/2584] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:18 2022] DELETE /identity/v3/users/5ddc3c84923644a79d8f536d504fcef1 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:19.035496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:19.036982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:19.051345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d6664-1a4b-4b6f-b929-26a4e13a2c18 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:19.110802 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:19.111377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:19.111658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:19.111935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] PATH_INFO: `/v3/projects/cc88772a0c534ab887caeab84baeb5fa` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:19.113060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc88772a0c534ab887caeab84baeb5fa)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:19.115008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:19.139256 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:19.268272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc88772a0c534ab887caeab84baeb5fa', 'request_id': 'req-259124b4-b043-4b28-93b0-2c15bb94f60b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:19.324989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc88772a0c534ab887caeab84baeb5fa', 'request_id': 'req-259124b4-b043-4b28-93b0-2c15bb94f60b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:19.331082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc88772a0c534ab887caeab84baeb5fa', 'request_id': 'req-259124b4-b043-4b28-93b0-2c15bb94f60b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:19.342392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] The token cache is being invalidate because project cc88772a0c534ab887caeab84baeb5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:56:19.343277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:19.343823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-259124b4-b043-4b28-93b0-2c15bb94f60b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:19.345656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1286/2585] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:56:19 2022] DELETE /identity/v3/projects/cc88772a0c534ab887caeab84baeb5fa => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:19.350543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f183d3-7cd6-47db-9c91-4a2d4f28f56d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:19.351028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f183d3-7cd6-47db-9c91-4a2d4f28f56d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:19.351142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f183d3-7cd6-47db-9c91-4a2d4f28f56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:19.371349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-74f183d3-7cd6-47db-9c91-4a2d4f28f56d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:19.372480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-74f183d3-7cd6-47db-9c91-4a2d4f28f56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:19.431179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1300/2586] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:19.439496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40715dc-fad0-4a71-a54e-713c1d7345fd None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:19.439781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40715dc-fad0-4a71-a54e-713c1d7345fd None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:19.440057 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40715dc-fad0-4a71-a54e-713c1d7345fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:19.459602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e40715dc-fad0-4a71-a54e-713c1d7345fd None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:19.460796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e40715dc-fad0-4a71-a54e-713c1d7345fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:19.469233 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1287/2587] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:19.473794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f183d3-7cd6-47db-9c91-4a2d4f28f56d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:19.483903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f183d3-7cd6-47db-9c91-4a2d4f28f56d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:19.488612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4d4ae-0662-4ccf-89a0-960f9f1a9065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:19.489204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4d4ae-0662-4ccf-89a0-960f9f1a9065 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:19.489484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4d4ae-0662-4ccf-89a0-960f9f1a9065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:19.489787 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4d4ae-0662-4ccf-89a0-960f9f1a9065 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:19.490430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4d4ae-0662-4ccf-89a0-960f9f1a9065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:56:19.490736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4d4ae-0662-4ccf-89a0-960f9f1a9065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:19.492346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4d4ae-0662-4ccf-89a0-960f9f1a9065 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:19.504786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1301/2588] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:56:19.509053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40715dc-fad0-4a71-a54e-713c1d7345fd None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:19.519216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40715dc-fad0-4a71-a54e-713c1d7345fd None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:19.523907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a86db-6b93-4a7c-810f-5bf9a4cc0df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:19.524506 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a86db-6b93-4a7c-810f-5bf9a4cc0df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:19.524787 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a86db-6b93-4a7c-810f-5bf9a4cc0df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:19.525064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a86db-6b93-4a7c-810f-5bf9a4cc0df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:19.525702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a86db-6b93-4a7c-810f-5bf9a4cc0df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:19.527693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a86db-6b93-4a7c-810f-5bf9a4cc0df5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:19.622442 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1288/2589] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:56:19 2022] POST /identity/v3/projects => generated 361 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:56:19.626448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4d4ae-0662-4ccf-89a0-960f9f1a9065 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:19.636422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4d4ae-0662-4ccf-89a0-960f9f1a9065 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:19.692639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2e407-adaf-4873-8790-d03759885458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:19.693235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2e407-adaf-4873-8790-d03759885458 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:19.693514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2e407-adaf-4873-8790-d03759885458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:19.693822 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2e407-adaf-4873-8790-d03759885458 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:19.694457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2e407-adaf-4873-8790-d03759885458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:19.696378 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2e407-adaf-4873-8790-d03759885458 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:19.799806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1302/2590] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:56:19 2022] POST /identity/v3/users => generated 281 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:19.803844 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a86db-6b93-4a7c-810f-5bf9a4cc0df5 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:19.815190 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a86db-6b93-4a7c-810f-5bf9a4cc0df5 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:19.820021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccbaeb-9c40-4395-b801-3bbf11c9f8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:19.820640 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccbaeb-9c40-4395-b801-3bbf11c9f8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:19.820914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccbaeb-9c40-4395-b801-3bbf11c9f8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:19.821183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccbaeb-9c40-4395-b801-3bbf11c9f8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:19.821851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccbaeb-9c40-4395-b801-3bbf11c9f8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:19.823441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccbaeb-9c40-4395-b801-3bbf11c9f8b4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:19.833713 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1289/2591] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:56:19.837689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2e407-adaf-4873-8790-d03759885458 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:19.848058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2e407-adaf-4873-8790-d03759885458 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:19.852788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f6508-34ba-4fd3-bdce-4eab95c8ec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:19.853400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f6508-34ba-4fd3-bdce-4eab95c8ec0b admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:19.853709 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f6508-34ba-4fd3-bdce-4eab95c8ec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:19.854082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f6508-34ba-4fd3-bdce-4eab95c8ec0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:19.854731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f6508-34ba-4fd3-bdce-4eab95c8ec0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:19.856357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f6508-34ba-4fd3-bdce-4eab95c8ec0b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:19.857636 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:19.868614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1303/2592] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:56:19.872781 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccbaeb-9c40-4395-b801-3bbf11c9f8b4 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:19.883516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccbaeb-9c40-4395-b801-3bbf11c9f8b4 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:19.888094 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86353138-5e00-4173-a6ef-a6cdb06056e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:19.888728 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86353138-5e00-4173-a6ef-a6cdb06056e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:19.889003 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86353138-5e00-4173-a6ef-a6cdb06056e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:19.889272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86353138-5e00-4173-a6ef-a6cdb06056e4 admin admin] PATH_INFO: `/v3/projects/301a0b2d34ac4c0f8a9ce32eeffeaae9/users/92e26d54a6d24ebcb483e77db59a1ff5/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:19.903573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86353138-5e00-4173-a6ef-a6cdb06056e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=301a0b2d34ac4c0f8a9ce32eeffeaae9, user_id=92e26d54a6d24ebcb483e77db59a1ff5, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:19.906477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86353138-5e00-4173-a6ef-a6cdb06056e4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:19.999855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1290/2593] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:19 2022] PUT /identity/v3/projects/301a0b2d34ac4c0f8a9ce32eeffeaae9/users/92e26d54a6d24ebcb483e77db59a1ff5/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:20.011390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf166fe3-8d71-4db2-8c14-c0d0da8bbfa2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:20.011676 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf166fe3-8d71-4db2-8c14-c0d0da8bbfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:20.012682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf166fe3-8d71-4db2-8c14-c0d0da8bbfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:20.041575 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-bf166fe3-8d71-4db2-8c14-c0d0da8bbfa2 None None] MFA Rules not processed for user `92e26d54a6d24ebcb483e77db59a1ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:20.042834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bf166fe3-8d71-4db2-8c14-c0d0da8bbfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:20.110967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1304/2594] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:20.148227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86353138-5e00-4173-a6ef-a6cdb06056e4 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:20.149994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-86353138-5e00-4173-a6ef-a6cdb06056e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:20.209015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86353138-5e00-4173-a6ef-a6cdb06056e4 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:20.212804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7006580-6bc3-4d5e-b4a0-8b3dd718bee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:20.213430 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006580-6bc3-4d5e-b4a0-8b3dd718bee8 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:20.213706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006580-6bc3-4d5e-b4a0-8b3dd718bee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:20.214026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006580-6bc3-4d5e-b4a0-8b3dd718bee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:20.222653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7006580-6bc3-4d5e-b4a0-8b3dd718bee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:20.224065 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7006580-6bc3-4d5e-b4a0-8b3dd718bee8 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:20.232784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1291/2595] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:20.248727 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf166fe3-8d71-4db2-8c14-c0d0da8bbfa2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:20.311245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf166fe3-8d71-4db2-8c14-c0d0da8bbfa2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:20.316747 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:20.317368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:20.317683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:20.318000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] PATH_INFO: `/v3/users/92e26d54a6d24ebcb483e77db59a1ff5` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:20.319123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e26d54a6d24ebcb483e77db59a1ff5)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:20.321034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:20.505017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e26d54a6d24ebcb483e77db59a1ff5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:20.517498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e26d54a6d24ebcb483e77db59a1ff5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:20.570991 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e26d54a6d24ebcb483e77db59a1ff5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:20.571620 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e26d54a6d24ebcb483e77db59a1ff5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:20.579517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e26d54a6d24ebcb483e77db59a1ff5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:20.585559 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1305/2596] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:20 2022] DELETE /identity/v3/users/92e26d54a6d24ebcb483e77db59a1ff5 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:20.590219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94274b-d77b-457d-ab17-76042a7f3c31 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:20.590505 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94274b-d77b-457d-ab17-76042a7f3c31 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:20.590785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94274b-d77b-457d-ab17-76042a7f3c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:20.610111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ca94274b-d77b-457d-ab17-76042a7f3c31 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:20.611315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ca94274b-d77b-457d-ab17-76042a7f3c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:20.655915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:20.669831 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1292/2597] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:20.684072 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:20.685518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:20.692480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:20.745798 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903f8e7-0ab5-4d96-b79f-ee08ebd6385a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:20.750406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2001df-7ec5-4cde-aa80-f2d4cf6a02f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:20.751015 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2001df-7ec5-4cde-aa80-f2d4cf6a02f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:20.752204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2001df-7ec5-4cde-aa80-f2d4cf6a02f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:20.752204 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2001df-7ec5-4cde-aa80-f2d4cf6a02f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:20.760761 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2001df-7ec5-4cde-aa80-f2d4cf6a02f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:20.762181 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2001df-7ec5-4cde-aa80-f2d4cf6a02f3 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:20.772166 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1306/2598] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:21.178943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94274b-d77b-457d-ab17-76042a7f3c31 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:21.186198 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ca94274b-d77b-457d-ab17-76042a7f3c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:21.196105 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94274b-d77b-457d-ab17-76042a7f3c31 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:21.200787 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:21.201421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:21.201698 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:21.202064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] PATH_INFO: `/v3/projects/301a0b2d34ac4c0f8a9ce32eeffeaae9` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:21.203519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] RBAC: Authorizing `identity:delete_project(project_id=301a0b2d34ac4c0f8a9ce32eeffeaae9)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:21.205446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:21.310557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '301a0b2d34ac4c0f8a9ce32eeffeaae9', 'request_id': 'req-68c05a03-584e-40c9-bfd2-f5f57cf8952c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:21.387851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '301a0b2d34ac4c0f8a9ce32eeffeaae9', 'request_id': 'req-68c05a03-584e-40c9-bfd2-f5f57cf8952c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:21.393874 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '301a0b2d34ac4c0f8a9ce32eeffeaae9', 'request_id': 'req-68c05a03-584e-40c9-bfd2-f5f57cf8952c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:21.404962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] The token cache is being invalidate because project 301a0b2d34ac4c0f8a9ce32eeffeaae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:56:21.405827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:21.406567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-68c05a03-584e-40c9-bfd2-f5f57cf8952c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:21.408063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1293/2599] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:56:21 2022] DELETE /identity/v3/projects/301a0b2d34ac4c0f8a9ce32eeffeaae9 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:21.412965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801abde3-5917-4213-a009-c848264931ea None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:21.413268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801abde3-5917-4213-a009-c848264931ea None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:21.413527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801abde3-5917-4213-a009-c848264931ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:21.433249 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-801abde3-5917-4213-a009-c848264931ea None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:21.434474 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-801abde3-5917-4213-a009-c848264931ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:21.494009 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1307/2600] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:21.501792 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73eef12-2f84-465e-9758-75dfcf3fe623 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:21.502079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73eef12-2f84-465e-9758-75dfcf3fe623 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:21.502351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73eef12-2f84-465e-9758-75dfcf3fe623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:21.521826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b73eef12-2f84-465e-9758-75dfcf3fe623 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:21.523004 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b73eef12-2f84-465e-9758-75dfcf3fe623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:21.531707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1294/2601] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:21.535566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801abde3-5917-4213-a009-c848264931ea None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:21.545294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801abde3-5917-4213-a009-c848264931ea None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:21.550740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcc9ec-0f7f-471c-b220-7ad47069b5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:21.551274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc9ec-0f7f-471c-b220-7ad47069b5eb admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:21.551553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc9ec-0f7f-471c-b220-7ad47069b5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:21.551820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc9ec-0f7f-471c-b220-7ad47069b5eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:21.552451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcc9ec-0f7f-471c-b220-7ad47069b5eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:56:21.552750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcc9ec-0f7f-471c-b220-7ad47069b5eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:21.554365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcc9ec-0f7f-471c-b220-7ad47069b5eb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:21.566222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1308/2602] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:56:21.570608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73eef12-2f84-465e-9758-75dfcf3fe623 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:21.579878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73eef12-2f84-465e-9758-75dfcf3fe623 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:21.584476 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b86e7-4409-4279-98ab-02b8e677457d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:21.585058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b86e7-4409-4279-98ab-02b8e677457d admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:21.585331 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b86e7-4409-4279-98ab-02b8e677457d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:21.585599 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b86e7-4409-4279-98ab-02b8e677457d admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:21.586245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b86e7-4409-4279-98ab-02b8e677457d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:21.588188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b86e7-4409-4279-98ab-02b8e677457d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:21.680521 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1295/2603] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:56:21 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:21.684752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcc9ec-0f7f-471c-b220-7ad47069b5eb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:21.694974 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcc9ec-0f7f-471c-b220-7ad47069b5eb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:21.751561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a80216-17b3-4ec6-b8d1-f684e499454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:21.752159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a80216-17b3-4ec6-b8d1-f684e499454b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:21.752439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a80216-17b3-4ec6-b8d1-f684e499454b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:21.752713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a80216-17b3-4ec6-b8d1-f684e499454b admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:21.753344 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a80216-17b3-4ec6-b8d1-f684e499454b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:21.755288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a80216-17b3-4ec6-b8d1-f684e499454b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:21.852690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1309/2604] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:56:21 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:21.856488 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b86e7-4409-4279-98ab-02b8e677457d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:21.867123 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b86e7-4409-4279-98ab-02b8e677457d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:21.871829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92251-1a5e-4ac5-96f9-f595d6e2bb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:21.877059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92251-1a5e-4ac5-96f9-f595d6e2bb04 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:21.877059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92251-1a5e-4ac5-96f9-f595d6e2bb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:21.877059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92251-1a5e-4ac5-96f9-f595d6e2bb04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:21.877059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd92251-1a5e-4ac5-96f9-f595d6e2bb04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:21.877059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd92251-1a5e-4ac5-96f9-f595d6e2bb04 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:21.885239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1296/2605] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:21.889217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a80216-17b3-4ec6-b8d1-f684e499454b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:21.899719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a80216-17b3-4ec6-b8d1-f684e499454b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:21.904264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48312b6b-0af1-4556-b747-a80a039a1601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:21.904842 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48312b6b-0af1-4556-b747-a80a039a1601 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:21.905750 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48312b6b-0af1-4556-b747-a80a039a1601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:21.906031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48312b6b-0af1-4556-b747-a80a039a1601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:21.906661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48312b6b-0af1-4556-b747-a80a039a1601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:21.908231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48312b6b-0af1-4556-b747-a80a039a1601 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:21.918643 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1310/2606] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:21.922651 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92251-1a5e-4ac5-96f9-f595d6e2bb04 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:21.932557 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92251-1a5e-4ac5-96f9-f595d6e2bb04 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:21.937673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b702d8-b14a-48bc-a2f0-e2f7cf3279ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:21.938464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b702d8-b14a-48bc-a2f0-e2f7cf3279ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:21.938796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b702d8-b14a-48bc-a2f0-e2f7cf3279ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:21.939083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b702d8-b14a-48bc-a2f0-e2f7cf3279ba admin admin] PATH_INFO: `/v3/projects/4362657a6b0349f3aaa5d8533a305364/users/8f4e16633eef4462918052748f2070df/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:21.954566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b702d8-b14a-48bc-a2f0-e2f7cf3279ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4362657a6b0349f3aaa5d8533a305364, user_id=8f4e16633eef4462918052748f2070df, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:21.957232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b702d8-b14a-48bc-a2f0-e2f7cf3279ba admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:22.039347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1297/2607] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:21 2022] PUT /identity/v3/projects/4362657a6b0349f3aaa5d8533a305364/users/8f4e16633eef4462918052748f2070df/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:22.056834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e739c-4bd1-4ba4-93a7-04c3c0889ce2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:22.057065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e739c-4bd1-4ba4-93a7-04c3c0889ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:22.057346 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e739c-4bd1-4ba4-93a7-04c3c0889ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:22.083019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:22.085851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-508e739c-4bd1-4ba4-93a7-04c3c0889ce2 None None] MFA Rules not processed for user `8f4e16633eef4462918052748f2070df`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:22.087064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-508e739c-4bd1-4ba4-93a7-04c3c0889ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:22.151944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1311/2608] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:22.163706 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b702d8-b14a-48bc-a2f0-e2f7cf3279ba admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:22.165413 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-10b702d8-b14a-48bc-a2f0-e2f7cf3279ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:22.222647 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b702d8-b14a-48bc-a2f0-e2f7cf3279ba admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:22.226265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bb14c-8b08-4968-ab07-f8c201e80779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:22.226855 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bb14c-8b08-4968-ab07-f8c201e80779 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:22.227128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bb14c-8b08-4968-ab07-f8c201e80779 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:22.227394 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bb14c-8b08-4968-ab07-f8c201e80779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:22.235791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bb14c-8b08-4968-ab07-f8c201e80779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:22.237199 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bb14c-8b08-4968-ab07-f8c201e80779 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:22.246018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1298/2609] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:32.149002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=26 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:32.242052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:38.180438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e739c-4bd1-4ba4-93a7-04c3c0889ce2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:38.182046 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-508e739c-4bd1-4ba4-93a7-04c3c0889ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:38.241421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e739c-4bd1-4ba4-93a7-04c3c0889ce2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:38.245314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd12e9a-f526-4c67-bd29-39767ca9264c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:38.245958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd12e9a-f526-4c67-bd29-39767ca9264c service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:38.246236 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd12e9a-f526-4c67-bd29-39767ca9264c service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:38.246508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd12e9a-f526-4c67-bd29-39767ca9264c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:38.248254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0bd12e9a-f526-4c67-bd29-39767ca9264c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:38.311852 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd12e9a-f526-4c67-bd29-39767ca9264c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:38.313283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd12e9a-f526-4c67-bd29-39767ca9264c service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:38.323105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1312/2610] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:41.901389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bb14c-8b08-4968-ab07-f8c201e80779 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:41.902775 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7c0bb14c-8b08-4968-ab07-f8c201e80779 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:41.996669 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bb14c-8b08-4968-ab07-f8c201e80779 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:41.999922 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4441678-6af9-4cd4-9573-0f35a11c861a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ec5c9c71d14762a4869b7d66e87ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:42.000524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441678-6af9-4cd4-9573-0f35a11c861a service placement] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:42.001197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441678-6af9-4cd4-9573-0f35a11c861a service placement] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:42.001478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441678-6af9-4cd4-9573-0f35a11c861a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:42.003238 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b4441678-6af9-4cd4-9573-0f35a11c861a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:42.022392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4441678-6af9-4cd4-9573-0f35a11c861a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:42.024281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4441678-6af9-4cd4-9573-0f35a11c861a service placement] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:42.037333 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1299/2611] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:42.082450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd12e9a-f526-4c67-bd29-39767ca9264c service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:42.096997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0bd12e9a-f526-4c67-bd29-39767ca9264c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:42.101679 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd12e9a-f526-4c67-bd29-39767ca9264c service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:42.106515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:42.107142 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:42.107424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:42.107701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] PATH_INFO: `/v3/users/2723e5c74bde49f485099b615b36997a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:42.109508 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2723e5c74bde49f485099b615b36997a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:42.112600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:42.298124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2723e5c74bde49f485099b615b36997a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:42.312895 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2723e5c74bde49f485099b615b36997a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:42.415243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2723e5c74bde49f485099b615b36997a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:42.416035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2723e5c74bde49f485099b615b36997a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:42.423617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-803bb4cf-88fd-4026-95f2-4fbecf02c178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2723e5c74bde49f485099b615b36997a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:42.430135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1313/2612] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:42 2022] DELETE /identity/v3/users/2723e5c74bde49f485099b615b36997a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:42.680449 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4441678-6af9-4cd4-9573-0f35a11c861a service placement] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:42.703010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4441678-6af9-4cd4-9573-0f35a11c861a service placement] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:42.776539 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:42.777379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:42.777672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:42.777983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] PATH_INFO: `/v3/projects/c1a07eadb5ea4914a6e1e6f3844afe94` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:42.779380 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a07eadb5ea4914a6e1e6f3844afe94)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:42.781541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:42.980207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a07eadb5ea4914a6e1e6f3844afe94', 'request_id': 'req-53991057-5ecf-478d-bb5a-a1e5c6859dee'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:43.062157 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a07eadb5ea4914a6e1e6f3844afe94', 'request_id': 'req-53991057-5ecf-478d-bb5a-a1e5c6859dee'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:43.068946 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a07eadb5ea4914a6e1e6f3844afe94', 'request_id': 'req-53991057-5ecf-478d-bb5a-a1e5c6859dee'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:43.080282 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] The token cache is being invalidate because project c1a07eadb5ea4914a6e1e6f3844afe94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:56:43.080570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:43.081223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-53991057-5ecf-478d-bb5a-a1e5c6859dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:43.083361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1300/2613] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:56:42 2022] DELETE /identity/v3/projects/c1a07eadb5ea4914a6e1e6f3844afe94 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:43.088342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dddb1-3cef-4599-a62e-c81c91b2bd4c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:43.088840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dddb1-3cef-4599-a62e-c81c91b2bd4c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:43.088944 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dddb1-3cef-4599-a62e-c81c91b2bd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:43.118209 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:43.139692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a17dddb1-3cef-4599-a62e-c81c91b2bd4c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:43.140913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a17dddb1-3cef-4599-a62e-c81c91b2bd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:43.203167 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1314/2614] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:43.233350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90243da-c74a-4a01-b2bf-3796f496893e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:43.233637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90243da-c74a-4a01-b2bf-3796f496893e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:43.233981 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90243da-c74a-4a01-b2bf-3796f496893e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:43.255359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-b90243da-c74a-4a01-b2bf-3796f496893e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:43.256581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-b90243da-c74a-4a01-b2bf-3796f496893e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:43.264982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1301/2615] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:56:43.269330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dddb1-3cef-4599-a62e-c81c91b2bd4c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:43.279188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dddb1-3cef-4599-a62e-c81c91b2bd4c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:43.283825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909bf9a-a9c4-444a-8f06-c4f849143436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:43.284420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909bf9a-a9c4-444a-8f06-c4f849143436 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:43.284696 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909bf9a-a9c4-444a-8f06-c4f849143436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:43.284994 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909bf9a-a9c4-444a-8f06-c4f849143436 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:43.285627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909bf9a-a9c4-444a-8f06-c4f849143436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:56:43.285957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909bf9a-a9c4-444a-8f06-c4f849143436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:43.287539 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909bf9a-a9c4-444a-8f06-c4f849143436 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:43.298975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1315/2616] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:56:43.305529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90243da-c74a-4a01-b2bf-3796f496893e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:43.313689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90243da-c74a-4a01-b2bf-3796f496893e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:43.319483 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9935d4-4c18-40e1-82c9-bead1d0d380e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:43.320152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9935d4-4c18-40e1-82c9-bead1d0d380e admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:43.320428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9935d4-4c18-40e1-82c9-bead1d0d380e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:43.320696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9935d4-4c18-40e1-82c9-bead1d0d380e admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:43.322663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9935d4-4c18-40e1-82c9-bead1d0d380e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:43.323809 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9935d4-4c18-40e1-82c9-bead1d0d380e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:43.336043 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:43.421678 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1302/2617] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:56:43 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:43.422981 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909bf9a-a9c4-444a-8f06-c4f849143436 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:43.433900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909bf9a-a9c4-444a-8f06-c4f849143436 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:43.500644 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92ef61-db8a-4d14-a604-a5c4b59d71ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:43.501292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92ef61-db8a-4d14-a604-a5c4b59d71ac admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:43.501583 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92ef61-db8a-4d14-a604-a5c4b59d71ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:43.501911 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92ef61-db8a-4d14-a604-a5c4b59d71ac admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:43.502585 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92ef61-db8a-4d14-a604-a5c4b59d71ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:43.504630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92ef61-db8a-4d14-a604-a5c4b59d71ac admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:43.571488 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1316/2618] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:56:43 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:43.575581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9935d4-4c18-40e1-82c9-bead1d0d380e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:43.586428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9935d4-4c18-40e1-82c9-bead1d0d380e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:43.591256 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68508a39-9ade-407d-a0af-8add8a5fee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:43.591871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68508a39-9ade-407d-a0af-8add8a5fee1c admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:43.592147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68508a39-9ade-407d-a0af-8add8a5fee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:43.592494 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68508a39-9ade-407d-a0af-8add8a5fee1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:43.593115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68508a39-9ade-407d-a0af-8add8a5fee1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:43.594756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68508a39-9ade-407d-a0af-8add8a5fee1c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:43.605586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1303/2619] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:43.609622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92ef61-db8a-4d14-a604-a5c4b59d71ac admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:43.620857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92ef61-db8a-4d14-a604-a5c4b59d71ac admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:43.625454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02948225-1124-413a-b9f3-cc09ea451018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:43.626076 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02948225-1124-413a-b9f3-cc09ea451018 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:43.626354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02948225-1124-413a-b9f3-cc09ea451018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:43.626623 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02948225-1124-413a-b9f3-cc09ea451018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:43.627259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02948225-1124-413a-b9f3-cc09ea451018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:43.629282 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02948225-1124-413a-b9f3-cc09ea451018 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:43.640005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1317/2620] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:43.644193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68508a39-9ade-407d-a0af-8add8a5fee1c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:43.654670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68508a39-9ade-407d-a0af-8add8a5fee1c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:43.659426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58ab84-0521-47a7-a07c-0abc76884f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:43.660696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58ab84-0521-47a7-a07c-0abc76884f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:43.661205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58ab84-0521-47a7-a07c-0abc76884f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:43.661543 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58ab84-0521-47a7-a07c-0abc76884f6d admin admin] PATH_INFO: `/v3/projects/838a2f4acd284628a2211575beecf1f6/users/c61880a6da6846d6bc32f803d108e458/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:43.678139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58ab84-0521-47a7-a07c-0abc76884f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=838a2f4acd284628a2211575beecf1f6, user_id=c61880a6da6846d6bc32f803d108e458, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:43.680800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58ab84-0521-47a7-a07c-0abc76884f6d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:43.759624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1304/2621] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:43 2022] PUT /identity/v3/projects/838a2f4acd284628a2211575beecf1f6/users/c61880a6da6846d6bc32f803d108e458/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:43.771126 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e66b12-7442-48e4-b6a3-d2bb63c6ba00 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:43.771413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e66b12-7442-48e4-b6a3-d2bb63c6ba00 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:43.771711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e66b12-7442-48e4-b6a3-d2bb63c6ba00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:43.810561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d5e66b12-7442-48e4-b6a3-d2bb63c6ba00 None None] MFA Rules not processed for user `c61880a6da6846d6bc32f803d108e458`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:43.818265 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d5e66b12-7442-48e4-b6a3-d2bb63c6ba00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:43.875651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:43.886388 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1318/2622] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:56:43.897997 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58ab84-0521-47a7-a07c-0abc76884f6d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:43.899657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1b58ab84-0521-47a7-a07c-0abc76884f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:44.012281 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58ab84-0521-47a7-a07c-0abc76884f6d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:44.018480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d176335-22d7-4af6-936d-247f23ea0b6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:44.019080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d176335-22d7-4af6-936d-247f23ea0b6c service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:44.019577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d176335-22d7-4af6-936d-247f23ea0b6c service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:44.019888 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d176335-22d7-4af6-936d-247f23ea0b6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:44.028690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d176335-22d7-4af6-936d-247f23ea0b6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:44.030417 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d176335-22d7-4af6-936d-247f23ea0b6c service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:44.039799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1305/2623] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:46.641930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e66b12-7442-48e4-b6a3-d2bb63c6ba00 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:46.643376 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d5e66b12-7442-48e4-b6a3-d2bb63c6ba00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:46.714757 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e66b12-7442-48e4-b6a3-d2bb63c6ba00 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:46.719177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:46.719797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:46.720335 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:46.720604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] PATH_INFO: `/v3/users/8f4e16633eef4462918052748f2070df` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:46.721739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f4e16633eef4462918052748f2070df)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:46.724185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:46.871772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f4e16633eef4462918052748f2070df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:46.885993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f4e16633eef4462918052748f2070df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:46.952065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f4e16633eef4462918052748f2070df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:46.952723 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f4e16633eef4462918052748f2070df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:46.961106 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f4e16633eef4462918052748f2070df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:46.967211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1319/2624] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:46 2022] DELETE /identity/v3/users/8f4e16633eef4462918052748f2070df => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:46.972211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dab4e-fabc-4013-8e4d-7eb1ce56fa0f None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:46.972779 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dab4e-fabc-4013-8e4d-7eb1ce56fa0f None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:46.973070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dab4e-fabc-4013-8e4d-7eb1ce56fa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:46.993789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-3d9dab4e-fabc-4013-8e4d-7eb1ce56fa0f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:46.994988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-3d9dab4e-fabc-4013-8e4d-7eb1ce56fa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:47.052695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1306/2625] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:47.062379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:47.066837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:47.122420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe369f-6428-40fd-8a96-55a473dd4216 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:47.126146 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838226dd-1786-4675-85c2-5df9cd82b259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:47.126751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838226dd-1786-4675-85c2-5df9cd82b259 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:47.127053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838226dd-1786-4675-85c2-5df9cd82b259 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:47.127314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838226dd-1786-4675-85c2-5df9cd82b259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:47.136426 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838226dd-1786-4675-85c2-5df9cd82b259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:47.137872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838226dd-1786-4675-85c2-5df9cd82b259 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:47.147876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1320/2626] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:47.532302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dab4e-fabc-4013-8e4d-7eb1ce56fa0f None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:47.534928 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-3d9dab4e-fabc-4013-8e4d-7eb1ce56fa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:47.548961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dab4e-fabc-4013-8e4d-7eb1ce56fa0f None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:47.553758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:47.555180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:47.555461 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:47.555780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] PATH_INFO: `/v3/projects/4362657a6b0349f3aaa5d8533a305364` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:47.556984 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4362657a6b0349f3aaa5d8533a305364)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:47.558839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:47.570301 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:47.662325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4362657a6b0349f3aaa5d8533a305364', 'request_id': 'req-b5c71198-5215-498c-ae73-faad6ca00588'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:47.723307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4362657a6b0349f3aaa5d8533a305364', 'request_id': 'req-b5c71198-5215-498c-ae73-faad6ca00588'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:47.729000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4362657a6b0349f3aaa5d8533a305364', 'request_id': 'req-b5c71198-5215-498c-ae73-faad6ca00588'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:47.740789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] The token cache is being invalidate because project 4362657a6b0349f3aaa5d8533a305364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:56:47.741072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:47.741694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b5c71198-5215-498c-ae73-faad6ca00588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:47.743188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1307/2627] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:56:47 2022] DELETE /identity/v3/projects/4362657a6b0349f3aaa5d8533a305364 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:47.748071 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c150c6-c077-45c8-b90d-31bae8d511bf None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:47.748549 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c150c6-c077-45c8-b90d-31bae8d511bf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:47.748665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c150c6-c077-45c8-b90d-31bae8d511bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:47.768557 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-47c150c6-c077-45c8-b90d-31bae8d511bf None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:47.769933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-47c150c6-c077-45c8-b90d-31bae8d511bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:47.779504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:47.829803 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1321/2628] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:47.837871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9044dc-b702-470e-8cac-afad52b3b604 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:47.838158 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9044dc-b702-470e-8cac-afad52b3b604 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:47.838456 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9044dc-b702-470e-8cac-afad52b3b604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:47.858406 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-9f9044dc-b702-470e-8cac-afad52b3b604 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:47.859616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9f9044dc-b702-470e-8cac-afad52b3b604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:47.868183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1308/2629] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:47.872384 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c150c6-c077-45c8-b90d-31bae8d511bf None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:47.882423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c150c6-c077-45c8-b90d-31bae8d511bf None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:47.886976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18811495-dcf6-42a3-a43b-fc9d434ca91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:47.887568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811495-dcf6-42a3-a43b-fc9d434ca91a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:47.887844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811495-dcf6-42a3-a43b-fc9d434ca91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:47.888113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811495-dcf6-42a3-a43b-fc9d434ca91a admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:47.888749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18811495-dcf6-42a3-a43b-fc9d434ca91a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:56:47.889048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18811495-dcf6-42a3-a43b-fc9d434ca91a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:47.890651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18811495-dcf6-42a3-a43b-fc9d434ca91a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:47.902501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1322/2630] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:56:47.906170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9044dc-b702-470e-8cac-afad52b3b604 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:47.915511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9044dc-b702-470e-8cac-afad52b3b604 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:47.920219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe632a-c56f-415a-abc9-d8656f4b5a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:47.920803 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe632a-c56f-415a-abc9-d8656f4b5a25 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:47.921106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe632a-c56f-415a-abc9-d8656f4b5a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:47.921392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe632a-c56f-415a-abc9-d8656f4b5a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:47.922050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe632a-c56f-415a-abc9-d8656f4b5a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:47.923943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe632a-c56f-415a-abc9-d8656f4b5a25 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:48.040796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1309/2631] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:56:47 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:48.045119 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18811495-dcf6-42a3-a43b-fc9d434ca91a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:48.055946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18811495-dcf6-42a3-a43b-fc9d434ca91a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:48.112382 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1063fc-54e9-4fb3-b368-ec82e5ec567d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:48.113690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1063fc-54e9-4fb3-b368-ec82e5ec567d admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:48.113976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1063fc-54e9-4fb3-b368-ec82e5ec567d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:48.114241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1063fc-54e9-4fb3-b368-ec82e5ec567d admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:48.114858 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1063fc-54e9-4fb3-b368-ec82e5ec567d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:48.116749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1063fc-54e9-4fb3-b368-ec82e5ec567d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:48.198418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1323/2632] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:56:48 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:48.202514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe632a-c56f-415a-abc9-d8656f4b5a25 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:48.213227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe632a-c56f-415a-abc9-d8656f4b5a25 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:48.217851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7170f98-19c6-4242-bea2-f49dafa86420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:48.218436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7170f98-19c6-4242-bea2-f49dafa86420 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:48.218712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7170f98-19c6-4242-bea2-f49dafa86420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:48.218979 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7170f98-19c6-4242-bea2-f49dafa86420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:48.219606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7170f98-19c6-4242-bea2-f49dafa86420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:48.221515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7170f98-19c6-4242-bea2-f49dafa86420 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:48.232347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1310/2633] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:48.236366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1063fc-54e9-4fb3-b368-ec82e5ec567d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:48.247640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1063fc-54e9-4fb3-b368-ec82e5ec567d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:48.252272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819feb49-49f4-49f4-a54d-b833145f28cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:48.252857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819feb49-49f4-49f4-a54d-b833145f28cf admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:48.253137 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819feb49-49f4-49f4-a54d-b833145f28cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:48.253413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819feb49-49f4-49f4-a54d-b833145f28cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:48.254497 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819feb49-49f4-49f4-a54d-b833145f28cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:48.256080 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819feb49-49f4-49f4-a54d-b833145f28cf admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:48.266545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1324/2634] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:48.270767 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7170f98-19c6-4242-bea2-f49dafa86420 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:48.280982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7170f98-19c6-4242-bea2-f49dafa86420 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:48.286337 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701b4c-4ed2-4252-9027-3f31db307894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:48.286985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701b4c-4ed2-4252-9027-3f31db307894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:48.287262 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701b4c-4ed2-4252-9027-3f31db307894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:48.287536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701b4c-4ed2-4252-9027-3f31db307894 admin admin] PATH_INFO: `/v3/projects/0f985324471c4f9da153fc4b5ae5c0e2/users/b7a68c8547eb4baf9c2d773b5d32dd4a/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:48.302736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701b4c-4ed2-4252-9027-3f31db307894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f985324471c4f9da153fc4b5ae5c0e2, user_id=b7a68c8547eb4baf9c2d773b5d32dd4a, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:48.305469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701b4c-4ed2-4252-9027-3f31db307894 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:48.383515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1311/2635] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:48 2022] PUT /identity/v3/projects/0f985324471c4f9da153fc4b5ae5c0e2/users/b7a68c8547eb4baf9c2d773b5d32dd4a/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:48.394078 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe1f00-8a0c-41f3-a1db-8ec2be6409ce None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:48.395624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe1f00-8a0c-41f3-a1db-8ec2be6409ce None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:48.395907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe1f00-8a0c-41f3-a1db-8ec2be6409ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:48.426300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-71fe1f00-8a0c-41f3-a1db-8ec2be6409ce None None] MFA Rules not processed for user `b7a68c8547eb4baf9c2d773b5d32dd4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:48.427594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-71fe1f00-8a0c-41f3-a1db-8ec2be6409ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:48.493553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1325/2636] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:48.505905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701b4c-4ed2-4252-9027-3f31db307894 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:48.507824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d4701b4c-4ed2-4252-9027-3f31db307894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:48.566291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701b4c-4ed2-4252-9027-3f31db307894 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:48.570263 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a73ae-a14d-4faa-9240-da3c3a69b7fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab54ee52004746eaadb5c2970e59afc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:48.570864 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a73ae-a14d-4faa-9240-da3c3a69b7fc service glance] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:48.575294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a73ae-a14d-4faa-9240-da3c3a69b7fc service glance] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:48.575294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a73ae-a14d-4faa-9240-da3c3a69b7fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:48.579800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a73ae-a14d-4faa-9240-da3c3a69b7fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:48.581270 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a73ae-a14d-4faa-9240-da3c3a69b7fc service glance] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:48.591125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1312/2637] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:52.228883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe1f00-8a0c-41f3-a1db-8ec2be6409ce None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:52.228883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-71fe1f00-8a0c-41f3-a1db-8ec2be6409ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:52.242849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:52.296333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe1f00-8a0c-41f3-a1db-8ec2be6409ce None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:52.301639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:52.302253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:52.302536 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:52.302814 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] PATH_INFO: `/v3/users/c61880a6da6846d6bc32f803d108e458` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:52.303979 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c61880a6da6846d6bc32f803d108e458)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:52.306018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:52.470632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c61880a6da6846d6bc32f803d108e458'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:52.489240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c61880a6da6846d6bc32f803d108e458'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:52.546806 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c61880a6da6846d6bc32f803d108e458'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:52.547439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c61880a6da6846d6bc32f803d108e458'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:52.558677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c61880a6da6846d6bc32f803d108e458'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:52.561647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1326/2638] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:52 2022] DELETE /identity/v3/users/c61880a6da6846d6bc32f803d108e458 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:52.566148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f960c-0221-4ff7-8e67-f8609443c899 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:52.566878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f960c-0221-4ff7-8e67-f8609443c899 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:52.566878 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f960c-0221-4ff7-8e67-f8609443c899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:52.589122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-0a1f960c-0221-4ff7-8e67-f8609443c899 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:52.590407 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0a1f960c-0221-4ff7-8e67-f8609443c899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:52.623410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:52.650088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1313/2639] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:56:52.661318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:52.662931 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:52.725077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d316383-3b0f-4b63-901c-ba9baf03def8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:52.729169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a18930-f6a2-4339-aa29-6b8bc99af038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:52.729834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a18930-f6a2-4339-aa29-6b8bc99af038 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:52.730114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a18930-f6a2-4339-aa29-6b8bc99af038 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:52.730406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a18930-f6a2-4339-aa29-6b8bc99af038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:52.740998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a18930-f6a2-4339-aa29-6b8bc99af038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:52.742468 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a18930-f6a2-4339-aa29-6b8bc99af038 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:52.752790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1327/2640] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:53.196048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f960c-0221-4ff7-8e67-f8609443c899 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:53.197491 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0a1f960c-0221-4ff7-8e67-f8609443c899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:53.213911 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f960c-0221-4ff7-8e67-f8609443c899 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:53.218949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:53.223007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:53.223007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:53.223007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] PATH_INFO: `/v3/projects/838a2f4acd284628a2211575beecf1f6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:53.223007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] RBAC: Authorizing `identity:delete_project(project_id=838a2f4acd284628a2211575beecf1f6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:53.224558 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:53.319935 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '838a2f4acd284628a2211575beecf1f6', 'request_id': 'req-83478586-7a9f-4bac-a764-dc283bf2a7db'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:53.376283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '838a2f4acd284628a2211575beecf1f6', 'request_id': 'req-83478586-7a9f-4bac-a764-dc283bf2a7db'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:53.382818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '838a2f4acd284628a2211575beecf1f6', 'request_id': 'req-83478586-7a9f-4bac-a764-dc283bf2a7db'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:53.394389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] The token cache is being invalidate because project 838a2f4acd284628a2211575beecf1f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:56:53.394673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:53.395327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-83478586-7a9f-4bac-a764-dc283bf2a7db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:53.398111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1314/2641] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:56:53 2022] DELETE /identity/v3/projects/838a2f4acd284628a2211575beecf1f6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:53.404778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a3226-edcd-47be-8d9c-84d1a1262f2b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:53.404778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a3226-edcd-47be-8d9c-84d1a1262f2b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:53.404778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a3226-edcd-47be-8d9c-84d1a1262f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:53.426017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7b0a3226-edcd-47be-8d9c-84d1a1262f2b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:53.427464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7b0a3226-edcd-47be-8d9c-84d1a1262f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:53.457101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:53.490825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1328/2642] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:56:53.499171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088b580-df39-4947-b6a4-6d2bf4c32be8 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:53.499455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088b580-df39-4947-b6a4-6d2bf4c32be8 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:53.499758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088b580-df39-4947-b6a4-6d2bf4c32be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:53.521490 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6088b580-df39-4947-b6a4-6d2bf4c32be8 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:53.522745 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6088b580-df39-4947-b6a4-6d2bf4c32be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:53.532286 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1315/2643] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:53.537087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a3226-edcd-47be-8d9c-84d1a1262f2b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:53.546819 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a3226-edcd-47be-8d9c-84d1a1262f2b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:53.551898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568350f9-f40c-4510-8a84-d9e6566c819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:53.554089 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568350f9-f40c-4510-8a84-d9e6566c819a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:53.554382 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568350f9-f40c-4510-8a84-d9e6566c819a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:53.554727 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568350f9-f40c-4510-8a84-d9e6566c819a admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:53.555362 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568350f9-f40c-4510-8a84-d9e6566c819a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:56:53.555681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568350f9-f40c-4510-8a84-d9e6566c819a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:53.557470 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568350f9-f40c-4510-8a84-d9e6566c819a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:53.575367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1329/2644] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 03:56:53.576566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088b580-df39-4947-b6a4-6d2bf4c32be8 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:53.587422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088b580-df39-4947-b6a4-6d2bf4c32be8 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:53.592318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653ced7-34f3-45bd-aba7-5ef9ed34d0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:53.592920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653ced7-34f3-45bd-aba7-5ef9ed34d0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:53.593195 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653ced7-34f3-45bd-aba7-5ef9ed34d0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:53.593467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653ced7-34f3-45bd-aba7-5ef9ed34d0f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:53.594156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653ced7-34f3-45bd-aba7-5ef9ed34d0f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:53.596356 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653ced7-34f3-45bd-aba7-5ef9ed34d0f1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:53.721412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1316/2645] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:56:53 2022] POST /identity/v3/projects => generated 363 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:56:53.726400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568350f9-f40c-4510-8a84-d9e6566c819a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:53.736504 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568350f9-f40c-4510-8a84-d9e6566c819a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:53.794463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf246faa-4e08-42d2-a055-ebdc5c748752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:53.795074 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf246faa-4e08-42d2-a055-ebdc5c748752 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:53.795358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf246faa-4e08-42d2-a055-ebdc5c748752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:53.795635 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf246faa-4e08-42d2-a055-ebdc5c748752 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:53.796266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf246faa-4e08-42d2-a055-ebdc5c748752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:53.798232 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf246faa-4e08-42d2-a055-ebdc5c748752 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:53.884534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1330/2646] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:56:53 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:56:53.885746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653ced7-34f3-45bd-aba7-5ef9ed34d0f1 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:53.897447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653ced7-34f3-45bd-aba7-5ef9ed34d0f1 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:53.902197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b1b7a-b8f8-4532-8efe-244e6db1aa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:53.902791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b1b7a-b8f8-4532-8efe-244e6db1aa19 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:53.903093 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b1b7a-b8f8-4532-8efe-244e6db1aa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:53.903361 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b1b7a-b8f8-4532-8efe-244e6db1aa19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:53.903992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b1b7a-b8f8-4532-8efe-244e6db1aa19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:53.905585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b1b7a-b8f8-4532-8efe-244e6db1aa19 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:53.916297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1317/2647] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:53.920454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf246faa-4e08-42d2-a055-ebdc5c748752 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:53.931172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf246faa-4e08-42d2-a055-ebdc5c748752 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:53.936147 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117500af-a3fb-46c5-8dd8-9ea6d6b040c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:53.936752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117500af-a3fb-46c5-8dd8-9ea6d6b040c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:53.937035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117500af-a3fb-46c5-8dd8-9ea6d6b040c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:53.937315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117500af-a3fb-46c5-8dd8-9ea6d6b040c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:53.937992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117500af-a3fb-46c5-8dd8-9ea6d6b040c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:53.939630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117500af-a3fb-46c5-8dd8-9ea6d6b040c1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:53.952185 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1331/2648] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:53.956264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b1b7a-b8f8-4532-8efe-244e6db1aa19 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:53.966973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b1b7a-b8f8-4532-8efe-244e6db1aa19 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:53.971914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805f17a-3a05-42b2-a76c-8eec64e90685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:53.972586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805f17a-3a05-42b2-a76c-8eec64e90685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:53.973019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805f17a-3a05-42b2-a76c-8eec64e90685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:53.973290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805f17a-3a05-42b2-a76c-8eec64e90685 admin admin] PATH_INFO: `/v3/projects/a77832e22eff4bc28ae124af9155aa69/users/639d94d4763e4897ae6252228a32d255/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:53.989064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805f17a-3a05-42b2-a76c-8eec64e90685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a77832e22eff4bc28ae124af9155aa69, user_id=639d94d4763e4897ae6252228a32d255, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:53.991785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805f17a-3a05-42b2-a76c-8eec64e90685 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:54.062601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1318/2649] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:53 2022] PUT /identity/v3/projects/a77832e22eff4bc28ae124af9155aa69/users/639d94d4763e4897ae6252228a32d255/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:54.074065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc3ea-4e74-4da8-9ddc-b0cedae0a1b8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:54.074356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc3ea-4e74-4da8-9ddc-b0cedae0a1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:54.074637 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc3ea-4e74-4da8-9ddc-b0cedae0a1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:54.105641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6affc3ea-4e74-4da8-9ddc-b0cedae0a1b8 None None] MFA Rules not processed for user `639d94d4763e4897ae6252228a32d255`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:54.106898 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6affc3ea-4e74-4da8-9ddc-b0cedae0a1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:54.172952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1332/2650] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:54.177570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805f17a-3a05-42b2-a76c-8eec64e90685 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:54.238075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:54.244436 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805f17a-3a05-42b2-a76c-8eec64e90685 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:54.252228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18629e5b-0f68-46f3-a99e-09ed195dac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:54.252960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629e5b-0f68-46f3-a99e-09ed195dac3b admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:54.253181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629e5b-0f68-46f3-a99e-09ed195dac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:54.253448 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629e5b-0f68-46f3-a99e-09ed195dac3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:54.254786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18629e5b-0f68-46f3-a99e-09ed195dac3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:54.256925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18629e5b-0f68-46f3-a99e-09ed195dac3b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:54.333701 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1319/2651] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:56:54 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:54.337857 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affc3ea-4e74-4da8-9ddc-b0cedae0a1b8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:54.349053 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affc3ea-4e74-4da8-9ddc-b0cedae0a1b8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:54.375129 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:54.407396 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e222154-016b-4250-8dd3-287875ec424e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:54.408097 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e222154-016b-4250-8dd3-287875ec424e admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:54.408271 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e222154-016b-4250-8dd3-287875ec424e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:54.408543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e222154-016b-4250-8dd3-287875ec424e admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:54.409169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e222154-016b-4250-8dd3-287875ec424e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:54.411216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e222154-016b-4250-8dd3-287875ec424e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:54.486678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1333/2652] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:56:54 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:56:54.491108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18629e5b-0f68-46f3-a99e-09ed195dac3b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:54.502039 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18629e5b-0f68-46f3-a99e-09ed195dac3b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:54.507773 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ea5bc-5ca8-464e-9292-10d81c00d20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:54.508349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ea5bc-5ca8-464e-9292-10d81c00d20c admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:54.508627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ea5bc-5ca8-464e-9292-10d81c00d20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:54.508903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ea5bc-5ca8-464e-9292-10d81c00d20c admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:54.509538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ea5bc-5ca8-464e-9292-10d81c00d20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:54.511169 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ea5bc-5ca8-464e-9292-10d81c00d20c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:54.521502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1320/2653] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:54.525653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e222154-016b-4250-8dd3-287875ec424e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:54.536004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e222154-016b-4250-8dd3-287875ec424e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:54.540657 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa0a62-a054-4150-a176-48d9208f61dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:54.541295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa0a62-a054-4150-a176-48d9208f61dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:54.541577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa0a62-a054-4150-a176-48d9208f61dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:54.541888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa0a62-a054-4150-a176-48d9208f61dc admin admin] PATH_INFO: `/v3/projects/5e9bc0e2fe7149e8a7715f6c768d9b91/users/eb94ebc6ec6246caa02ac5cf19728b13/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:54.560867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa0a62-a054-4150-a176-48d9208f61dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9bc0e2fe7149e8a7715f6c768d9b91, user_id=eb94ebc6ec6246caa02ac5cf19728b13, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:54.563652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa0a62-a054-4150-a176-48d9208f61dc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:54.635309 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1334/2654] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:54 2022] PUT /identity/v3/projects/5e9bc0e2fe7149e8a7715f6c768d9b91/users/eb94ebc6ec6246caa02ac5cf19728b13/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:56:54.640578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ea5bc-5ca8-464e-9292-10d81c00d20c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:54.650707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ea5bc-5ca8-464e-9292-10d81c00d20c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:54.711648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bee39-d028-465f-932f-371aadd59a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:54.712659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bee39-d028-465f-932f-371aadd59a4b admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:54.713156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bee39-d028-465f-932f-371aadd59a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:54.713646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bee39-d028-465f-932f-371aadd59a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:54.714441 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bee39-d028-465f-932f-371aadd59a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:54.716031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bee39-d028-465f-932f-371aadd59a4b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:54.726837 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1321/2655] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:56:54.732215 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa0a62-a054-4150-a176-48d9208f61dc admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:54.748375 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa0a62-a054-4150-a176-48d9208f61dc admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:54.749997 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96b383-2000-49df-b435-23a434f40da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:54.750645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96b383-2000-49df-b435-23a434f40da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:54.750922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96b383-2000-49df-b435-23a434f40da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:54.751193 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96b383-2000-49df-b435-23a434f40da1 admin admin] PATH_INFO: `/v3/projects/5e9bc0e2fe7149e8a7715f6c768d9b91/users/eb94ebc6ec6246caa02ac5cf19728b13/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:54.753262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96b383-2000-49df-b435-23a434f40da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9bc0e2fe7149e8a7715f6c768d9b91, user_id=eb94ebc6ec6246caa02ac5cf19728b13, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:54.756169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96b383-2000-49df-b435-23a434f40da1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:54.766826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1335/2656] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:56:54 2022] PUT /identity/v3/projects/5e9bc0e2fe7149e8a7715f6c768d9b91/users/eb94ebc6ec6246caa02ac5cf19728b13/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:54.780362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d50eaa-1b85-4280-b469-c184a84f7208 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:54.780756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d50eaa-1b85-4280-b469-c184a84f7208 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:54.781060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d50eaa-1b85-4280-b469-c184a84f7208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:54.813078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-32d50eaa-1b85-4280-b469-c184a84f7208 None None] MFA Rules not processed for user `eb94ebc6ec6246caa02ac5cf19728b13`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:54.814310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-32d50eaa-1b85-4280-b469-c184a84f7208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:54.883539 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1322/2657] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:54.897964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96b383-2000-49df-b435-23a434f40da1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:54.897964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cf96b383-2000-49df-b435-23a434f40da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:54.955697 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96b383-2000-49df-b435-23a434f40da1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:54.959523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d754a3c-e77e-4d92-bfb9-613c5e5cb507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:54.960318 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d754a3c-e77e-4d92-bfb9-613c5e5cb507 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:54.960704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d754a3c-e77e-4d92-bfb9-613c5e5cb507 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:54.961087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d754a3c-e77e-4d92-bfb9-613c5e5cb507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:54.977496 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d754a3c-e77e-4d92-bfb9-613c5e5cb507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:54.978983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d754a3c-e77e-4d92-bfb9-613c5e5cb507 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:54.989207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1336/2658] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:55.184219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d50eaa-1b85-4280-b469-c184a84f7208 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:55.194574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d50eaa-1b85-4280-b469-c184a84f7208 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:55.198067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee598a6-0b0a-4014-a2a8-aacd1021ee47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:55.198657 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee598a6-0b0a-4014-a2a8-aacd1021ee47 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:55.203791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee598a6-0b0a-4014-a2a8-aacd1021ee47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:55.203791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee598a6-0b0a-4014-a2a8-aacd1021ee47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:55.251232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:55.260896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee598a6-0b0a-4014-a2a8-aacd1021ee47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:55.262404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee598a6-0b0a-4014-a2a8-aacd1021ee47 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:55.271311 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1323/2659] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:58.656186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d754a3c-e77e-4d92-bfb9-613c5e5cb507 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:58.725162 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d754a3c-e77e-4d92-bfb9-613c5e5cb507 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:58.730745 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:58.731393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:58.731872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:58.731872 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] PATH_INFO: `/v3/users/639d94d4763e4897ae6252228a32d255` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:58.733382 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=639d94d4763e4897ae6252228a32d255)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:58.735389 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:58.812153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:58.931081 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '639d94d4763e4897ae6252228a32d255'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:58.943486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '639d94d4763e4897ae6252228a32d255'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:59.056360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '639d94d4763e4897ae6252228a32d255'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:59.056921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '639d94d4763e4897ae6252228a32d255'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:59.064711 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '639d94d4763e4897ae6252228a32d255'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:59.071647 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1337/2660] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:58 2022] DELETE /identity/v3/users/639d94d4763e4897ae6252228a32d255 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:59.077536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0b140-47cd-40a8-826a-33d8d813cc56 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:59.078119 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0b140-47cd-40a8-826a-33d8d813cc56 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:59.078240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0b140-47cd-40a8-826a-33d8d813cc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:59.100124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-dfa0b140-47cd-40a8-826a-33d8d813cc56 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:56:59.101893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-dfa0b140-47cd-40a8-826a-33d8d813cc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:59.160976 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1324/2661] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:56:59.171722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:59.173131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:59.229952 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c8db0-5f2c-49f7-b11b-16f371d8b692 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:59.235220 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda7f91-e7c3-4109-a0e2-704a87826f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:59.236023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda7f91-e7c3-4109-a0e2-704a87826f73 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:59.236351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda7f91-e7c3-4109-a0e2-704a87826f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:59.236629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda7f91-e7c3-4109-a0e2-704a87826f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:59.247107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda7f91-e7c3-4109-a0e2-704a87826f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:59.248525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda7f91-e7c3-4109-a0e2-704a87826f73 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:59.258136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1338/2662] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:59.309567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0b140-47cd-40a8-826a-33d8d813cc56 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:59.310919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-dfa0b140-47cd-40a8-826a-33d8d813cc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:59.370546 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0b140-47cd-40a8-826a-33d8d813cc56 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:59.377240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fe6d2-7e8f-4aa5-8a05-99a6ee4c70d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:59.377240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fe6d2-7e8f-4aa5-8a05-99a6ee4c70d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:59.377240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fe6d2-7e8f-4aa5-8a05-99a6ee4c70d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:59.377240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fe6d2-7e8f-4aa5-8a05-99a6ee4c70d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:59.377921 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-856fe6d2-7e8f-4aa5-8a05-99a6ee4c70d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:59.440135 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fe6d2-7e8f-4aa5-8a05-99a6ee4c70d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:59.441656 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fe6d2-7e8f-4aa5-8a05-99a6ee4c70d7 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:59.450923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1325/2663] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:56:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:56:59.653277 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda7f91-e7c3-4109-a0e2-704a87826f73 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:59.654890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2eda7f91-e7c3-4109-a0e2-704a87826f73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:56:59.669715 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda7f91-e7c3-4109-a0e2-704a87826f73 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:59.674818 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:59.675445 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:59.675731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:59.676008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] PATH_INFO: `/v3/projects/a77832e22eff4bc28ae124af9155aa69` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:59.677583 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a77832e22eff4bc28ae124af9155aa69)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:59.679595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:56:59.789269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a77832e22eff4bc28ae124af9155aa69', 'request_id': 'req-b3787a92-a723-4965-9d88-0dbf8207d972'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:59.795534 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a77832e22eff4bc28ae124af9155aa69', 'request_id': 'req-b3787a92-a723-4965-9d88-0dbf8207d972'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:59.884976 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a77832e22eff4bc28ae124af9155aa69', 'request_id': 'req-b3787a92-a723-4965-9d88-0dbf8207d972'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:59.900884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] The token cache is being invalidate because project a77832e22eff4bc28ae124af9155aa69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:56:59.901127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:59.901921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:56:59.903398 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1339/2664] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:56:59 2022] DELETE /identity/v3/projects/a77832e22eff4bc28ae124af9155aa69 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:56:59.907489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fe6d2-7e8f-4aa5-8a05-99a6ee4c70d7 service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:56:59.918757 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fe6d2-7e8f-4aa5-8a05-99a6ee4c70d7 service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:56:59.967260 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:56:59.984471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:56:59.985101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:56:59.987148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:56:59.987148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] PATH_INFO: `/v3/users/eb94ebc6ec6246caa02ac5cf19728b13` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:56:59.987148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb94ebc6ec6246caa02ac5cf19728b13)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:56:59.988849 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:00.207692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb94ebc6ec6246caa02ac5cf19728b13'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:00.284862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb94ebc6ec6246caa02ac5cf19728b13'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:00.285486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb94ebc6ec6246caa02ac5cf19728b13'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:00.293154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb94ebc6ec6246caa02ac5cf19728b13'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:00.305868 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-b6bb6981-f2ef-49d4-8065-d61d37374413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb94ebc6ec6246caa02ac5cf19728b13'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:00.311850 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1326/2665] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:56:59 2022] DELETE /identity/v3/users/eb94ebc6ec6246caa02ac5cf19728b13 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:00.714713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:00.716170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:00.730305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3787a92-a723-4965-9d88-0dbf8207d972 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:00.758660 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:00.790616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:00.791246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:00.791528 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:00.791811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] PATH_INFO: `/v3/projects/5e9bc0e2fe7149e8a7715f6c768d9b91` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:00.792972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e9bc0e2fe7149e8a7715f6c768d9b91)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:00.794958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:00.902738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e9bc0e2fe7149e8a7715f6c768d9b91', 'request_id': 'req-0136c7c5-9d5e-402c-be8b-51c3b693ef53'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:00.908754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e9bc0e2fe7149e8a7715f6c768d9b91', 'request_id': 'req-0136c7c5-9d5e-402c-be8b-51c3b693ef53'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:00.986550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e9bc0e2fe7149e8a7715f6c768d9b91', 'request_id': 'req-0136c7c5-9d5e-402c-be8b-51c3b693ef53'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:00.999315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] The token cache is being invalidate because project 5e9bc0e2fe7149e8a7715f6c768d9b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:57:00.999603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:01.000254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0136c7c5-9d5e-402c-be8b-51c3b693ef53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:01.001912 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1340/2666] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:57:00 2022] DELETE /identity/v3/projects/5e9bc0e2fe7149e8a7715f6c768d9b91 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:01.006707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb284f-7179-412a-8aa2-bec41ce92cd7 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.006995 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb284f-7179-412a-8aa2-bec41ce92cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.007315 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb284f-7179-412a-8aa2-bec41ce92cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.027753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-66fb284f-7179-412a-8aa2-bec41ce92cd7 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:01.028968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-66fb284f-7179-412a-8aa2-bec41ce92cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:01.087690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1327/2667] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:01.095873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d80be-50d6-4d71-bcc9-96e148c8eb71 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.096153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d80be-50d6-4d71-bcc9-96e148c8eb71 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.096427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d80be-50d6-4d71-bcc9-96e148c8eb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.117211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e58d80be-50d6-4d71-bcc9-96e148c8eb71 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:01.118459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e58d80be-50d6-4d71-bcc9-96e148c8eb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:01.126492 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1341/2668] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:01.130756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb284f-7179-412a-8aa2-bec41ce92cd7 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:01.140646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb284f-7179-412a-8aa2-bec41ce92cd7 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:01.145514 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dcd88-9d13-4122-9c7c-f876f5a4a541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:01.146287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dcd88-9d13-4122-9c7c-f876f5a4a541 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.146487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dcd88-9d13-4122-9c7c-f876f5a4a541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.146758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dcd88-9d13-4122-9c7c-f876f5a4a541 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.147423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dcd88-9d13-4122-9c7c-f876f5a4a541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:57:01.147727 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dcd88-9d13-4122-9c7c-f876f5a4a541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:01.150132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dcd88-9d13-4122-9c7c-f876f5a4a541 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:01.161707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1328/2669] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:57:01.167027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d80be-50d6-4d71-bcc9-96e148c8eb71 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:01.176270 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d80be-50d6-4d71-bcc9-96e148c8eb71 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:01.181356 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5adf4-2294-45ba-8c74-7cd22af514ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:01.181973 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5adf4-2294-45ba-8c74-7cd22af514ee admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.182256 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5adf4-2294-45ba-8c74-7cd22af514ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.182531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5adf4-2294-45ba-8c74-7cd22af514ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.183157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5adf4-2294-45ba-8c74-7cd22af514ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:01.185034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5adf4-2294-45ba-8c74-7cd22af514ee admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:01.268110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1342/2670] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:57:01 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:01.272302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dcd88-9d13-4122-9c7c-f876f5a4a541 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:01.283074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dcd88-9d13-4122-9c7c-f876f5a4a541 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:01.329580 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:01.338945 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4eeba-190b-4880-bd0a-527336ab282d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:01.339601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4eeba-190b-4880-bd0a-527336ab282d admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.339880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4eeba-190b-4880-bd0a-527336ab282d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.340151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4eeba-190b-4880-bd0a-527336ab282d admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.340769 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4eeba-190b-4880-bd0a-527336ab282d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:01.342720 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4eeba-190b-4880-bd0a-527336ab282d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:01.420341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1329/2671] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:57:01 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:01.424459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5adf4-2294-45ba-8c74-7cd22af514ee admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:01.435102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5adf4-2294-45ba-8c74-7cd22af514ee admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:01.439765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ed8cc-d270-4b67-9166-2ef4ac39f972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:01.440353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ed8cc-d270-4b67-9166-2ef4ac39f972 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.440632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ed8cc-d270-4b67-9166-2ef4ac39f972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.440907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ed8cc-d270-4b67-9166-2ef4ac39f972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.441543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ed8cc-d270-4b67-9166-2ef4ac39f972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:01.443221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ed8cc-d270-4b67-9166-2ef4ac39f972 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:01.453993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1343/2672] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:01.457806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4eeba-190b-4880-bd0a-527336ab282d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:01.468503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4eeba-190b-4880-bd0a-527336ab282d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:01.473400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1235c-4c4b-4b83-8b5b-2c1ed0a623d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:01.474013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1235c-4c4b-4b83-8b5b-2c1ed0a623d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.474292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1235c-4c4b-4b83-8b5b-2c1ed0a623d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.474561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1235c-4c4b-4b83-8b5b-2c1ed0a623d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.475187 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1235c-4c4b-4b83-8b5b-2c1ed0a623d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:01.477046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1235c-4c4b-4b83-8b5b-2c1ed0a623d7 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:01.487871 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1330/2673] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:01.491965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ed8cc-d270-4b67-9166-2ef4ac39f972 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:01.501718 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ed8cc-d270-4b67-9166-2ef4ac39f972 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:01.506245 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e007ec-96bd-402f-9726-37da5fbfc9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:01.506874 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e007ec-96bd-402f-9726-37da5fbfc9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.507152 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e007ec-96bd-402f-9726-37da5fbfc9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.508042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e007ec-96bd-402f-9726-37da5fbfc9f1 admin admin] PATH_INFO: `/v3/projects/3e6c9230d9ef4a6c93abaa763f481a8e/users/223830eb023d46a1b959e3609749620b/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.522773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e007ec-96bd-402f-9726-37da5fbfc9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e6c9230d9ef4a6c93abaa763f481a8e, user_id=223830eb023d46a1b959e3609749620b, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:01.526058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e007ec-96bd-402f-9726-37da5fbfc9f1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:01.593967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1344/2674] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:57:01 2022] PUT /identity/v3/projects/3e6c9230d9ef4a6c93abaa763f481a8e/users/223830eb023d46a1b959e3609749620b/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:01.605947 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae22069-7a6b-42e4-9786-2fc0eb06efa9 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.606229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae22069-7a6b-42e4-9786-2fc0eb06efa9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.606502 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae22069-7a6b-42e4-9786-2fc0eb06efa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.634577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-5ae22069-7a6b-42e4-9786-2fc0eb06efa9 None None] MFA Rules not processed for user `223830eb023d46a1b959e3609749620b`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:01.635815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5ae22069-7a6b-42e4-9786-2fc0eb06efa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:01.704776 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1331/2675] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:01.709293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e007ec-96bd-402f-9726-37da5fbfc9f1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:01.767464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:01.772438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e007ec-96bd-402f-9726-37da5fbfc9f1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:01.778595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3ce11-695e-4d78-8761-e9e9ae83c4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:01.778595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3ce11-695e-4d78-8761-e9e9ae83c4ad admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.778595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3ce11-695e-4d78-8761-e9e9ae83c4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.778595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3ce11-695e-4d78-8761-e9e9ae83c4ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.779068 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3ce11-695e-4d78-8761-e9e9ae83c4ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:01.780802 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3ce11-695e-4d78-8761-e9e9ae83c4ad admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:01.868777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1345/2676] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:57:01 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:01.872518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae22069-7a6b-42e4-9786-2fc0eb06efa9 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:01.884010 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae22069-7a6b-42e4-9786-2fc0eb06efa9 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:01.939796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75386b1-9549-45f3-b809-eb6d21a55cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:01.940399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75386b1-9549-45f3-b809-eb6d21a55cbd admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:01.940677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75386b1-9549-45f3-b809-eb6d21a55cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:01.940954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75386b1-9549-45f3-b809-eb6d21a55cbd admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:01.941580 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75386b1-9549-45f3-b809-eb6d21a55cbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:01.943528 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75386b1-9549-45f3-b809-eb6d21a55cbd admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:02.018207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1332/2677] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:57:01 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:57:02.022070 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3ce11-695e-4d78-8761-e9e9ae83c4ad admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:02.034443 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3ce11-695e-4d78-8761-e9e9ae83c4ad admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:02.038035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab40332-35da-498e-b8d4-a47a31fa3236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:02.038725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40332-35da-498e-b8d4-a47a31fa3236 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:02.039011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40332-35da-498e-b8d4-a47a31fa3236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:02.039291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40332-35da-498e-b8d4-a47a31fa3236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:02.039938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab40332-35da-498e-b8d4-a47a31fa3236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:02.043531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab40332-35da-498e-b8d4-a47a31fa3236 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:02.057992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1346/2678] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:57:02.065797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75386b1-9549-45f3-b809-eb6d21a55cbd admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:02.076559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75386b1-9549-45f3-b809-eb6d21a55cbd admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:02.081139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7862d-5a4a-4e6e-850c-73c0a20c6414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:02.081835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7862d-5a4a-4e6e-850c-73c0a20c6414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:02.082115 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7862d-5a4a-4e6e-850c-73c0a20c6414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:02.082391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7862d-5a4a-4e6e-850c-73c0a20c6414 admin admin] PATH_INFO: `/v3/projects/ee4f1540b1e44d1eb74b306f8ef46398/users/96d3b44bf08a4cd391b384cbc950a861/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:02.097585 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7862d-5a4a-4e6e-850c-73c0a20c6414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4f1540b1e44d1eb74b306f8ef46398, user_id=96d3b44bf08a4cd391b384cbc950a861, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:02.100292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7862d-5a4a-4e6e-850c-73c0a20c6414 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:02.169104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1333/2679] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:57:02 2022] PUT /identity/v3/projects/ee4f1540b1e44d1eb74b306f8ef46398/users/96d3b44bf08a4cd391b384cbc950a861/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:57:02.173064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab40332-35da-498e-b8d4-a47a31fa3236 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:02.184390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab40332-35da-498e-b8d4-a47a31fa3236 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:02.240939 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57340bb-5da1-4562-a2c0-bca2fb724721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:02.241538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57340bb-5da1-4562-a2c0-bca2fb724721 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:02.242535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57340bb-5da1-4562-a2c0-bca2fb724721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:02.243035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57340bb-5da1-4562-a2c0-bca2fb724721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:02.243678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57340bb-5da1-4562-a2c0-bca2fb724721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:02.245267 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57340bb-5da1-4562-a2c0-bca2fb724721 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:02.255924 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1347/2680] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:02.258989 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7862d-5a4a-4e6e-850c-73c0a20c6414 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:02.269159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7862d-5a4a-4e6e-850c-73c0a20c6414 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:02.273714 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6b1a1-e673-41e6-85dc-cb95103c94f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:02.274390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6b1a1-e673-41e6-85dc-cb95103c94f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:02.274663 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6b1a1-e673-41e6-85dc-cb95103c94f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:02.274931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6b1a1-e673-41e6-85dc-cb95103c94f8 admin admin] PATH_INFO: `/v3/projects/ee4f1540b1e44d1eb74b306f8ef46398/users/96d3b44bf08a4cd391b384cbc950a861/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:02.276882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6b1a1-e673-41e6-85dc-cb95103c94f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4f1540b1e44d1eb74b306f8ef46398, user_id=96d3b44bf08a4cd391b384cbc950a861, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:02.279616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6b1a1-e673-41e6-85dc-cb95103c94f8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:02.292164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1334/2681] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:57:02 2022] PUT /identity/v3/projects/ee4f1540b1e44d1eb74b306f8ef46398/users/96d3b44bf08a4cd391b384cbc950a861/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:02.301171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d27d-e7da-40b3-b147-eadd69437647 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:02.301473 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d27d-e7da-40b3-b147-eadd69437647 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:02.301765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d27d-e7da-40b3-b147-eadd69437647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:02.329427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-58a3d27d-e7da-40b3-b147-eadd69437647 None None] MFA Rules not processed for user `96d3b44bf08a4cd391b384cbc950a861`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:02.330639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-58a3d27d-e7da-40b3-b147-eadd69437647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:02.405195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1348/2682] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:02.418047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6b1a1-e673-41e6-85dc-cb95103c94f8 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:02.419682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a3b6b1a1-e673-41e6-85dc-cb95103c94f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:02.434958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:02.476125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6b1a1-e673-41e6-85dc-cb95103c94f8 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:02.480881 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bfcfb-3ad1-4d0a-aaa1-45b958e4d8bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:02.481471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bfcfb-3ad1-4d0a-aaa1-45b958e4d8bc service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:02.481789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bfcfb-3ad1-4d0a-aaa1-45b958e4d8bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:02.482037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bfcfb-3ad1-4d0a-aaa1-45b958e4d8bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:02.490953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bfcfb-3ad1-4d0a-aaa1-45b958e4d8bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:02.492399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bfcfb-3ad1-4d0a-aaa1-45b958e4d8bc service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:02.501797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1335/2683] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:03.441617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3d27d-e7da-40b3-b147-eadd69437647 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:03.451312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3d27d-e7da-40b3-b147-eadd69437647 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:03.456158 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73257a1-dc41-4c85-9db9-18e2d2997918 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d3b44bf08a4cd391b384cbc950a861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee4f1540b1e44d1eb74b306f8ef46398', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:03.456781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73257a1-dc41-4c85-9db9-18e2d2997918 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:03.457067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73257a1-dc41-4c85-9db9-18e2d2997918 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:03.457351 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73257a1-dc41-4c85-9db9-18e2d2997918 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:03.458029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73257a1-dc41-4c85-9db9-18e2d2997918 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:57:03.458334 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73257a1-dc41-4c85-9db9-18e2d2997918 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:03.459970 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73257a1-dc41-4c85-9db9-18e2d2997918 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:03.472550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1349/2684] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:57:03.477482 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bfcfb-3ad1-4d0a-aaa1-45b958e4d8bc service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:03.487170 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bfcfb-3ad1-4d0a-aaa1-45b958e4d8bc service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:03.492018 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a500e-3649-4d5d-9d29-cbde7e5133cb tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d3b44bf08a4cd391b384cbc950a861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee4f1540b1e44d1eb74b306f8ef46398', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:03.493432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a500e-3649-4d5d-9d29-cbde7e5133cb tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:03.493432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a500e-3649-4d5d-9d29-cbde7e5133cb tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:03.493432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a500e-3649-4d5d-9d29-cbde7e5133cb tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:03.494006 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a500e-3649-4d5d-9d29-cbde7e5133cb tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:03.495904 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a500e-3649-4d5d-9d29-cbde7e5133cb tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:03.589446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1336/2685] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:57:03 2022] POST /identity/v3/projects => generated 353 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:03.593820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73257a1-dc41-4c85-9db9-18e2d2997918 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:03.605018 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73257a1-dc41-4c85-9db9-18e2d2997918 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:03.639722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:03.663484 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbeebd-2ea4-4770-a06e-f69cf25b8221 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d3b44bf08a4cd391b384cbc950a861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee4f1540b1e44d1eb74b306f8ef46398', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:03.664342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbeebd-2ea4-4770-a06e-f69cf25b8221 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:03.664493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbeebd-2ea4-4770-a06e-f69cf25b8221 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:03.664763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbeebd-2ea4-4770-a06e-f69cf25b8221 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:03.665397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbeebd-2ea4-4770-a06e-f69cf25b8221 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:57:03.665700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbeebd-2ea4-4770-a06e-f69cf25b8221 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:03.667424 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbeebd-2ea4-4770-a06e-f69cf25b8221 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:03.679577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1350/2686] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:57:03.718798 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a500e-3649-4d5d-9d29-cbde7e5133cb tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:03.730067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a500e-3649-4d5d-9d29-cbde7e5133cb tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:03.734910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d3b44bf08a4cd391b384cbc950a861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee4f1540b1e44d1eb74b306f8ef46398', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:03.737223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:03.737223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:03.737223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] PATH_INFO: `/v3/projects/afb76a1f9de34a2a964b6e697b1f2d79` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:03.737564 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: Authorizing `identity:delete_project(project_id=afb76a1f9de34a2a964b6e697b1f2d79)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:03.741015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:03.834834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb76a1f9de34a2a964b6e697b1f2d79', 'request_id': 'req-f4997bfc-dc00-4829-83e9-14d9f0626471'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:03.902185 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb76a1f9de34a2a964b6e697b1f2d79', 'request_id': 'req-f4997bfc-dc00-4829-83e9-14d9f0626471'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:03.907982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb76a1f9de34a2a964b6e697b1f2d79', 'request_id': 'req-f4997bfc-dc00-4829-83e9-14d9f0626471'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:03.919833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] The token cache is being invalidate because project afb76a1f9de34a2a964b6e697b1f2d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:57:03.920052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:03.920691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-f4997bfc-dc00-4829-83e9-14d9f0626471 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:03.922209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1337/2687] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:57:03 2022] DELETE /identity/v3/projects/afb76a1f9de34a2a964b6e697b1f2d79 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:04.574238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbeebd-2ea4-4770-a06e-f69cf25b8221 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:04.576120 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-57dbeebd-2ea4-4770-a06e-f69cf25b8221 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:04.666221 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbeebd-2ea4-4770-a06e-f69cf25b8221 tempest-VolumeQuotaClassesTest-370272635 tempest-VolumeQuotaClassesTest-370272635-project-admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:04.742536 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:04.743218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:04.743500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:04.743915 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] PATH_INFO: `/v3/users/223830eb023d46a1b959e3609749620b` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:04.745088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=223830eb023d46a1b959e3609749620b)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:04.747138 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:04.924774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '223830eb023d46a1b959e3609749620b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:04.937404 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '223830eb023d46a1b959e3609749620b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:05.028094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '223830eb023d46a1b959e3609749620b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:05.028744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '223830eb023d46a1b959e3609749620b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:05.037014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '223830eb023d46a1b959e3609749620b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:05.039187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:05.045744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1351/2688] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:57:04 2022] DELETE /identity/v3/users/223830eb023d46a1b959e3609749620b => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:05.050523 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47838e3-6f16-4513-bbd0-801946108dea None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:05.051131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47838e3-6f16-4513-bbd0-801946108dea None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:05.051264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47838e3-6f16-4513-bbd0-801946108dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:05.071903 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-f47838e3-6f16-4513-bbd0-801946108dea None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:05.073592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f47838e3-6f16-4513-bbd0-801946108dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:05.148512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1338/2689] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:57:05.161607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:05.163117 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:05.231511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00141db8-7839-4562-9e17-e80a8d580fe8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:05.235165 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562ae16-e68d-4e61-92d4-f57189b8611d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:05.235783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562ae16-e68d-4e61-92d4-f57189b8611d service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:05.236061 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562ae16-e68d-4e61-92d4-f57189b8611d service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:05.236331 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562ae16-e68d-4e61-92d4-f57189b8611d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:05.246608 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562ae16-e68d-4e61-92d4-f57189b8611d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:05.248051 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562ae16-e68d-4e61-92d4-f57189b8611d service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:05.260367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1352/2690] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:05.709334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47838e3-6f16-4513-bbd0-801946108dea None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:05.710951 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-f47838e3-6f16-4513-bbd0-801946108dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:05.725454 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47838e3-6f16-4513-bbd0-801946108dea None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:05.729889 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:05.731218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:05.731508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:05.731784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] PATH_INFO: `/v3/projects/3e6c9230d9ef4a6c93abaa763f481a8e` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:05.732938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e6c9230d9ef4a6c93abaa763f481a8e)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:05.735029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:05.747912 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:05.881930 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e6c9230d9ef4a6c93abaa763f481a8e', 'request_id': 'req-c6818d11-5330-47c1-a3dc-0d58d2335022'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:05.941891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e6c9230d9ef4a6c93abaa763f481a8e', 'request_id': 'req-c6818d11-5330-47c1-a3dc-0d58d2335022'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:05.947778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e6c9230d9ef4a6c93abaa763f481a8e', 'request_id': 'req-c6818d11-5330-47c1-a3dc-0d58d2335022'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:05.958823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] The token cache is being invalidate because project 3e6c9230d9ef4a6c93abaa763f481a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:57:05.959110 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:05.959741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:05.961237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1339/2691] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:57:05 2022] DELETE /identity/v3/projects/3e6c9230d9ef4a6c93abaa763f481a8e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:57:05.965326 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562ae16-e68d-4e61-92d4-f57189b8611d service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:05.975227 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562ae16-e68d-4e61-92d4-f57189b8611d service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:06.032188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:06.032825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:06.033104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:06.033380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] PATH_INFO: `/v3/users/96d3b44bf08a4cd391b384cbc950a861` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:06.034530 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d3b44bf08a4cd391b384cbc950a861)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:06.036431 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:06.190522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d3b44bf08a4cd391b384cbc950a861'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:06.202687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d3b44bf08a4cd391b384cbc950a861'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:06.283413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d3b44bf08a4cd391b384cbc950a861'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:06.284032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d3b44bf08a4cd391b384cbc950a861'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:06.292641 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5b0a1f3c-c70b-4ff4-840b-a63b764acd29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d3b44bf08a4cd391b384cbc950a861'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:06.298670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1353/2692] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:57:05 2022] DELETE /identity/v3/users/96d3b44bf08a4cd391b384cbc950a861 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:06.715695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:06.717236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:06.732009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6818d11-5330-47c1-a3dc-0d58d2335022 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:06.798379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:06.799015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:06.799379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:06.799561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] PATH_INFO: `/v3/projects/ee4f1540b1e44d1eb74b306f8ef46398` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:06.800712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee4f1540b1e44d1eb74b306f8ef46398)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:06.802637 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:07.029161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee4f1540b1e44d1eb74b306f8ef46398', 'request_id': 'req-2dcb4858-118b-42c1-9336-70c7b9a1ddad'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:07.113499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee4f1540b1e44d1eb74b306f8ef46398', 'request_id': 'req-2dcb4858-118b-42c1-9336-70c7b9a1ddad'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:07.120391 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee4f1540b1e44d1eb74b306f8ef46398', 'request_id': 'req-2dcb4858-118b-42c1-9336-70c7b9a1ddad'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:07.132982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] The token cache is being invalidate because project ee4f1540b1e44d1eb74b306f8ef46398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:57:07.133265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:07.133943 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-2dcb4858-118b-42c1-9336-70c7b9a1ddad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:07.135458 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1340/2693] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:57:06 2022] DELETE /identity/v3/projects/ee4f1540b1e44d1eb74b306f8ef46398 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:07.140344 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55757356-6ab6-4379-86fe-849c56abe97b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:07.140848 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55757356-6ab6-4379-86fe-849c56abe97b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:07.140955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55757356-6ab6-4379-86fe-849c56abe97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:07.160982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-55757356-6ab6-4379-86fe-849c56abe97b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:07.162216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-55757356-6ab6-4379-86fe-849c56abe97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:07.175029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:07.221246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1354/2694] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:07.229613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde570a-cee3-4d03-b9c3-52389c6dc94b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:07.229957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde570a-cee3-4d03-b9c3-52389c6dc94b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:07.230228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde570a-cee3-4d03-b9c3-52389c6dc94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:07.251515 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ddde570a-cee3-4d03-b9c3-52389c6dc94b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:07.252746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ddde570a-cee3-4d03-b9c3-52389c6dc94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:07.261668 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1341/2695] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:07.265916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55757356-6ab6-4379-86fe-849c56abe97b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:07.276115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55757356-6ab6-4379-86fe-849c56abe97b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:07.280797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eebed-c6d9-4c18-aa3c-7a9f2c189563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:07.281342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eebed-c6d9-4c18-aa3c-7a9f2c189563 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:07.281627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eebed-c6d9-4c18-aa3c-7a9f2c189563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:07.281940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eebed-c6d9-4c18-aa3c-7a9f2c189563 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:07.282592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eebed-c6d9-4c18-aa3c-7a9f2c189563 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:57:07.282902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eebed-c6d9-4c18-aa3c-7a9f2c189563 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:07.284532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eebed-c6d9-4c18-aa3c-7a9f2c189563 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:07.300003 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1355/2696] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:57:07.301183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde570a-cee3-4d03-b9c3-52389c6dc94b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:07.311471 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde570a-cee3-4d03-b9c3-52389c6dc94b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:07.316056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f990c55-66bd-4111-806d-5b572f803fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:07.316646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f990c55-66bd-4111-806d-5b572f803fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:07.316919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f990c55-66bd-4111-806d-5b572f803fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:07.317184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f990c55-66bd-4111-806d-5b572f803fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:07.317827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f990c55-66bd-4111-806d-5b572f803fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:07.319724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f990c55-66bd-4111-806d-5b572f803fc6 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:07.405744 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1342/2697] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:57:07 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:07.410465 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eebed-c6d9-4c18-aa3c-7a9f2c189563 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:07.420958 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eebed-c6d9-4c18-aa3c-7a9f2c189563 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:07.478297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f21f6d-b892-439a-95ac-8b556d894d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:07.478890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f21f6d-b892-439a-95ac-8b556d894d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:07.479168 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f21f6d-b892-439a-95ac-8b556d894d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:07.479439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f21f6d-b892-439a-95ac-8b556d894d8b admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:07.480064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f21f6d-b892-439a-95ac-8b556d894d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:07.481968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f21f6d-b892-439a-95ac-8b556d894d8b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:07.557465 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1356/2698] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:57:07 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:07.567383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f990c55-66bd-4111-806d-5b572f803fc6 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:07.573359 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:07.578729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f990c55-66bd-4111-806d-5b572f803fc6 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:07.583338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349c18f-a7b4-4aae-957a-7c9ac4a66c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:07.583953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349c18f-a7b4-4aae-957a-7c9ac4a66c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:07.584232 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349c18f-a7b4-4aae-957a-7c9ac4a66c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:07.584504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349c18f-a7b4-4aae-957a-7c9ac4a66c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:07.585175 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349c18f-a7b4-4aae-957a-7c9ac4a66c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:07.587464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349c18f-a7b4-4aae-957a-7c9ac4a66c36 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:07.597988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1343/2699] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:07.602069 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f21f6d-b892-439a-95ac-8b556d894d8b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:07.611840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f21f6d-b892-439a-95ac-8b556d894d8b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:07.616760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc70450-5cf3-4829-a396-4c52f226576f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:07.617347 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc70450-5cf3-4829-a396-4c52f226576f admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:07.617630 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc70450-5cf3-4829-a396-4c52f226576f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:07.617935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc70450-5cf3-4829-a396-4c52f226576f admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:07.618577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc70450-5cf3-4829-a396-4c52f226576f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:07.620259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc70450-5cf3-4829-a396-4c52f226576f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:07.630773 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1357/2700] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:07.635445 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349c18f-a7b4-4aae-957a-7c9ac4a66c36 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:07.645219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349c18f-a7b4-4aae-957a-7c9ac4a66c36 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:07.650799 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962787d-d115-4cd6-8533-e0160f129755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:07.651437 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962787d-d115-4cd6-8533-e0160f129755 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:07.651717 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962787d-d115-4cd6-8533-e0160f129755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:07.652022 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962787d-d115-4cd6-8533-e0160f129755 admin admin] PATH_INFO: `/v3/projects/804febe354a24c40aadb6d2c0fdea4b9/users/0d3293ddf6e54b94afb53636548b9330/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:07.666885 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962787d-d115-4cd6-8533-e0160f129755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=804febe354a24c40aadb6d2c0fdea4b9, user_id=0d3293ddf6e54b94afb53636548b9330, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:07.669536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962787d-d115-4cd6-8533-e0160f129755 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:07.751366 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1344/2701] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:57:07 2022] PUT /identity/v3/projects/804febe354a24c40aadb6d2c0fdea4b9/users/0d3293ddf6e54b94afb53636548b9330/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:57:07.763259 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd245c5-7112-4187-86d0-7894a5946579 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:07.763551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd245c5-7112-4187-86d0-7894a5946579 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:07.763834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd245c5-7112-4187-86d0-7894a5946579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:07.791500 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2fd245c5-7112-4187-86d0-7894a5946579 None None] MFA Rules not processed for user `0d3293ddf6e54b94afb53636548b9330`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:07.792714 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2fd245c5-7112-4187-86d0-7894a5946579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:07.863439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1358/2702] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:07.868466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962787d-d115-4cd6-8533-e0160f129755 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:07.928415 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962787d-d115-4cd6-8533-e0160f129755 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:07.933664 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9f53a-ff22-4452-a371-252eaf98d5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:07.934291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9f53a-ff22-4452-a371-252eaf98d5ed admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:07.934572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9f53a-ff22-4452-a371-252eaf98d5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:07.934870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9f53a-ff22-4452-a371-252eaf98d5ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:07.935496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9f53a-ff22-4452-a371-252eaf98d5ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:07.937377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9f53a-ff22-4452-a371-252eaf98d5ed admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:08.029783 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1345/2703] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:57:07 2022] POST /identity/v3/projects => generated 389 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:08.033910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd245c5-7112-4187-86d0-7894a5946579 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:08.045274 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd245c5-7112-4187-86d0-7894a5946579 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:08.057666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:08.103386 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd4142-486f-47a6-94e5-0419da005928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:08.103916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4142-486f-47a6-94e5-0419da005928 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:08.104192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4142-486f-47a6-94e5-0419da005928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:08.104460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4142-486f-47a6-94e5-0419da005928 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:08.105077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd4142-486f-47a6-94e5-0419da005928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:08.107037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd4142-486f-47a6-94e5-0419da005928 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:08.187570 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1359/2704] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:57:08 2022] POST /identity/v3/users => generated 301 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:08.191654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9f53a-ff22-4452-a371-252eaf98d5ed admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:08.202368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9f53a-ff22-4452-a371-252eaf98d5ed admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:08.207403 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d0fcc-e0fb-43bb-87b4-b5aeb9e83d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:08.208000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d0fcc-e0fb-43bb-87b4-b5aeb9e83d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:08.208273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d0fcc-e0fb-43bb-87b4-b5aeb9e83d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:08.208541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d0fcc-e0fb-43bb-87b4-b5aeb9e83d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:08.209168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d0fcc-e0fb-43bb-87b4-b5aeb9e83d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:08.210814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d0fcc-e0fb-43bb-87b4-b5aeb9e83d86 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:08.221519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1346/2705] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:08.225864 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd4142-486f-47a6-94e5-0419da005928 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:08.236668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd4142-486f-47a6-94e5-0419da005928 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:08.241284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbae61-4b3e-40b1-adbc-069fe17f2e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:08.242595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbae61-4b3e-40b1-adbc-069fe17f2e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:08.242873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbae61-4b3e-40b1-adbc-069fe17f2e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:08.243144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbae61-4b3e-40b1-adbc-069fe17f2e64 admin admin] PATH_INFO: `/v3/projects/f4d7c980420a4acaacba9e3ca736c2ea/users/eed07532766c4681a4ebe811f32584b7/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:08.257861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fbae61-4b3e-40b1-adbc-069fe17f2e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d7c980420a4acaacba9e3ca736c2ea, user_id=eed07532766c4681a4ebe811f32584b7, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:08.260502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fbae61-4b3e-40b1-adbc-069fe17f2e64 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:08.317632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1360/2706] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:57:08 2022] PUT /identity/v3/projects/f4d7c980420a4acaacba9e3ca736c2ea/users/eed07532766c4681a4ebe811f32584b7/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 03:57:08.321821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d0fcc-e0fb-43bb-87b4-b5aeb9e83d86 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:08.332383 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d0fcc-e0fb-43bb-87b4-b5aeb9e83d86 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:08.390960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdd1bf-dfcd-4449-8616-036c2013781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:08.391560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdd1bf-dfcd-4449-8616-036c2013781c admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:08.391836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdd1bf-dfcd-4449-8616-036c2013781c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:08.392106 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdd1bf-dfcd-4449-8616-036c2013781c admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:08.392738 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdd1bf-dfcd-4449-8616-036c2013781c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:08.394381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdd1bf-dfcd-4449-8616-036c2013781c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:08.403968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1347/2707] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:08.408192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbae61-4b3e-40b1-adbc-069fe17f2e64 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:08.419442 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbae61-4b3e-40b1-adbc-069fe17f2e64 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:08.423983 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fc115-0601-4fdb-ae9e-ddb5cd4f3dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:08.424684 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fc115-0601-4fdb-ae9e-ddb5cd4f3dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:08.424966 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fc115-0601-4fdb-ae9e-ddb5cd4f3dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:08.425240 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fc115-0601-4fdb-ae9e-ddb5cd4f3dd2 admin admin] PATH_INFO: `/v3/projects/f4d7c980420a4acaacba9e3ca736c2ea/users/eed07532766c4681a4ebe811f32584b7/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:08.427163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fc115-0601-4fdb-ae9e-ddb5cd4f3dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d7c980420a4acaacba9e3ca736c2ea, user_id=eed07532766c4681a4ebe811f32584b7, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:08.430558 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fc115-0601-4fdb-ae9e-ddb5cd4f3dd2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:08.440699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1361/2708] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:57:08 2022] PUT /identity/v3/projects/f4d7c980420a4acaacba9e3ca736c2ea/users/eed07532766c4681a4ebe811f32584b7/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:08.450688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ede99-c704-451d-93de-83028afb51ed None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:08.450977 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ede99-c704-451d-93de-83028afb51ed None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:08.451283 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ede99-c704-451d-93de-83028afb51ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:08.479759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-9c5ede99-c704-451d-93de-83028afb51ed None None] MFA Rules not processed for user `eed07532766c4681a4ebe811f32584b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:08.480960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9c5ede99-c704-451d-93de-83028afb51ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:08.506587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:08.550012 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1348/2709] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:08.560938 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fc115-0601-4fdb-ae9e-ddb5cd4f3dd2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:08.562632 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fa6fc115-0601-4fdb-ae9e-ddb5cd4f3dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:08.621638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fc115-0601-4fdb-ae9e-ddb5cd4f3dd2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:08.625264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500dc26-0d67-4f30-a1d6-e7315e594d07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:08.625894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500dc26-0d67-4f30-a1d6-e7315e594d07 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:08.626177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500dc26-0d67-4f30-a1d6-e7315e594d07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:08.626461 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500dc26-0d67-4f30-a1d6-e7315e594d07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:08.635325 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500dc26-0d67-4f30-a1d6-e7315e594d07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:08.636752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500dc26-0d67-4f30-a1d6-e7315e594d07 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:08.646133 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1362/2710] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:08.796567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ede99-c704-451d-93de-83028afb51ed None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:08.858886 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ede99-c704-451d-93de-83028afb51ed None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:08.863339 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:08.863957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:08.864230 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:08.864498 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] PATH_INFO: `/v3/users/0d3293ddf6e54b94afb53636548b9330` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:08.865602 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3293ddf6e54b94afb53636548b9330)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:08.867619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:09.031319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3293ddf6e54b94afb53636548b9330'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:09.090724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3293ddf6e54b94afb53636548b9330'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:09.091373 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3293ddf6e54b94afb53636548b9330'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:09.101938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3293ddf6e54b94afb53636548b9330'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:09.115050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-98aba7f5-68c6-4968-a55a-7ce5efa22b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3293ddf6e54b94afb53636548b9330'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:09.121842 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1349/2711] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:57:08 2022] DELETE /identity/v3/users/0d3293ddf6e54b94afb53636548b9330 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:09.126366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a94c2-067e-4b8a-bdc0-ad07a06182e9 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:09.126657 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a94c2-067e-4b8a-bdc0-ad07a06182e9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:09.126932 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a94c2-067e-4b8a-bdc0-ad07a06182e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:09.133552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cce70-86fa-44fe-94fd-ccf0fb206068 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:09.133848 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cce70-86fa-44fe-94fd-ccf0fb206068 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:09.134128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cce70-86fa-44fe-94fd-ccf0fb206068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:09.146568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-164a94c2-067e-4b8a-bdc0-ad07a06182e9 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:09.147730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-164a94c2-067e-4b8a-bdc0-ad07a06182e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:09.153613 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-1b5cce70-86fa-44fe-94fd-ccf0fb206068 None None] MFA Rules not processed for user `89faad7b1c26412cac5c2c624fa3f0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:09.154847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1b5cce70-86fa-44fe-94fd-ccf0fb206068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:09.194035 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:09.209520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1363/2712] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:09.212425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1350/2713] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:57:09.224341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a94c2-067e-4b8a-bdc0-ad07a06182e9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:09.224341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-164a94c2-067e-4b8a-bdc0-ad07a06182e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:09.235815 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:09.282397 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a94c2-067e-4b8a-bdc0-ad07a06182e9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:09.286284 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559526e2-702b-4232-a638-1b6768519c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:09.286890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559526e2-702b-4232-a638-1b6768519c32 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:09.287169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559526e2-702b-4232-a638-1b6768519c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:09.289969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559526e2-702b-4232-a638-1b6768519c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:09.299891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559526e2-702b-4232-a638-1b6768519c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:09.301353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559526e2-702b-4232-a638-1b6768519c32 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:09.311783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1364/2714] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:09.871140 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cce70-86fa-44fe-94fd-ccf0fb206068 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:09.873378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-1b5cce70-86fa-44fe-94fd-ccf0fb206068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:09.890539 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cce70-86fa-44fe-94fd-ccf0fb206068 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:09.895817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:09.897147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:09.897147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:09.897369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] PATH_INFO: `/v3/projects/804febe354a24c40aadb6d2c0fdea4b9` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:09.901082 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=804febe354a24c40aadb6d2c0fdea4b9)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:09.902673 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:10.006078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804febe354a24c40aadb6d2c0fdea4b9', 'request_id': 'req-34c8e100-ecf8-48b0-9130-6cb8958e9da3'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:10.062159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804febe354a24c40aadb6d2c0fdea4b9', 'request_id': 'req-34c8e100-ecf8-48b0-9130-6cb8958e9da3'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:10.068046 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804febe354a24c40aadb6d2c0fdea4b9', 'request_id': 'req-34c8e100-ecf8-48b0-9130-6cb8958e9da3'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:10.079747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] The token cache is being invalidate because project 804febe354a24c40aadb6d2c0fdea4b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:57:10.080034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:10.080653 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:10.082184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1351/2715] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:57:09 2022] DELETE /identity/v3/projects/804febe354a24c40aadb6d2c0fdea4b9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:10.086425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559526e2-702b-4232-a638-1b6768519c32 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:10.096502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559526e2-702b-4232-a638-1b6768519c32 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:10.156796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:10.157429 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:10.157749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:10.158103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] PATH_INFO: `/v3/users/eed07532766c4681a4ebe811f32584b7` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:10.159243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed07532766c4681a4ebe811f32584b7)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:10.161189 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:10.317268 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed07532766c4681a4ebe811f32584b7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:10.334348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed07532766c4681a4ebe811f32584b7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:10.403311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed07532766c4681a4ebe811f32584b7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:10.404171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed07532766c4681a4ebe811f32584b7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:10.413070 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-72fa9421-37c1-4a17-897f-2bdf08e8e947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed07532766c4681a4ebe811f32584b7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:10.419936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1365/2716] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:57:10 2022] DELETE /identity/v3/users/eed07532766c4681a4ebe811f32584b7 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:10.873481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:10.875064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:10.889425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8e100-ecf8-48b0-9130-6cb8958e9da3 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:10.950277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:10.950908 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:10.951188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:10.951460 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] PATH_INFO: `/v3/projects/f4d7c980420a4acaacba9e3ca736c2ea` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:10.952582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d7c980420a4acaacba9e3ca736c2ea)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:10.954534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:11.063679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d7c980420a4acaacba9e3ca736c2ea', 'request_id': 'req-eb5ff8ae-01f8-4869-be63-a861e99971bd'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:11.176716 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d7c980420a4acaacba9e3ca736c2ea', 'request_id': 'req-eb5ff8ae-01f8-4869-be63-a861e99971bd'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:11.182931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d7c980420a4acaacba9e3ca736c2ea', 'request_id': 'req-eb5ff8ae-01f8-4869-be63-a861e99971bd'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:11.190808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:11.194549 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] The token cache is being invalidate because project f4d7c980420a4acaacba9e3ca736c2ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:57:11.195341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:11.196090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-eb5ff8ae-01f8-4869-be63-a861e99971bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:11.197826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1352/2717] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:57:10 2022] DELETE /identity/v3/projects/f4d7c980420a4acaacba9e3ca736c2ea => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:11.202521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653512f-f75b-42c6-be6b-c0112b9710e6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:11.202812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653512f-f75b-42c6-be6b-c0112b9710e6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:11.203091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653512f-f75b-42c6-be6b-c0112b9710e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:11.223992 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6653512f-f75b-42c6-be6b-c0112b9710e6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:11.225190 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6653512f-f75b-42c6-be6b-c0112b9710e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:11.256239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:11.289005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1366/2718] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:11.297289 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72032908-22a7-4016-8bd3-1cc2477cd46b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:11.297570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72032908-22a7-4016-8bd3-1cc2477cd46b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:11.297910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72032908-22a7-4016-8bd3-1cc2477cd46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:11.317475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-72032908-22a7-4016-8bd3-1cc2477cd46b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:11.318724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-72032908-22a7-4016-8bd3-1cc2477cd46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:11.327399 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1353/2719] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:11.331856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653512f-f75b-42c6-be6b-c0112b9710e6 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:11.342476 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653512f-f75b-42c6-be6b-c0112b9710e6 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:11.347289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fb027-0a88-4e96-972b-fa1d0a0bb710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:11.347891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fb027-0a88-4e96-972b-fa1d0a0bb710 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:11.348177 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fb027-0a88-4e96-972b-fa1d0a0bb710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:11.348456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fb027-0a88-4e96-972b-fa1d0a0bb710 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:11.349104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fb027-0a88-4e96-972b-fa1d0a0bb710 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:57:11.349413 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fb027-0a88-4e96-972b-fa1d0a0bb710 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:11.351159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fb027-0a88-4e96-972b-fa1d0a0bb710 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:11.362746 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1367/2720] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:57:11.367253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72032908-22a7-4016-8bd3-1cc2477cd46b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:11.377432 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72032908-22a7-4016-8bd3-1cc2477cd46b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:11.383174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab243401-3360-4c17-8170-e41fbbea28e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:11.383942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab243401-3360-4c17-8170-e41fbbea28e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:11.384323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab243401-3360-4c17-8170-e41fbbea28e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:11.384747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab243401-3360-4c17-8170-e41fbbea28e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:11.385497 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab243401-3360-4c17-8170-e41fbbea28e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:11.387685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab243401-3360-4c17-8170-e41fbbea28e1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:11.530405 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1354/2721] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:57:11 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:11.534813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fb027-0a88-4e96-972b-fa1d0a0bb710 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:11.545285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fb027-0a88-4e96-972b-fa1d0a0bb710 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:11.612313 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24c369-c7dc-4d22-885d-2dc25ff4f609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:11.613037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24c369-c7dc-4d22-885d-2dc25ff4f609 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:11.613316 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24c369-c7dc-4d22-885d-2dc25ff4f609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:11.613617 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24c369-c7dc-4d22-885d-2dc25ff4f609 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:11.614300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24c369-c7dc-4d22-885d-2dc25ff4f609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:11.616214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24c369-c7dc-4d22-885d-2dc25ff4f609 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:11.658346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab243401-3360-4c17-8170-e41fbbea28e1 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:11.661122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ab243401-3360-4c17-8170-e41fbbea28e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:11.724602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1368/2722] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:57:11 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 03:57:11.725744 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24c369-c7dc-4d22-885d-2dc25ff4f609 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:11.735826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24c369-c7dc-4d22-885d-2dc25ff4f609 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:11.741276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a778a6d-c2be-4135-a499-c938e99adb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:11.744033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a778a6d-c2be-4135-a499-c938e99adb61 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:11.744033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a778a6d-c2be-4135-a499-c938e99adb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:11.744033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a778a6d-c2be-4135-a499-c938e99adb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:11.744033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a778a6d-c2be-4135-a499-c938e99adb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:11.745555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a778a6d-c2be-4135-a499-c938e99adb61 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:11.753942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab243401-3360-4c17-8170-e41fbbea28e1 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:11.759921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1369/2723] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:11.760071 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628b308-1c39-4b9b-b76a-4a35e06b7305 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ec5c9c71d14762a4869b7d66e87ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:11.760679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628b308-1c39-4b9b-b76a-4a35e06b7305 service placement] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:11.760955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628b308-1c39-4b9b-b76a-4a35e06b7305 service placement] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:11.761217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628b308-1c39-4b9b-b76a-4a35e06b7305 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:11.762742 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a778a6d-c2be-4135-a499-c938e99adb61 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:11.763070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5628b308-1c39-4b9b-b76a-4a35e06b7305 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:11.772241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a778a6d-c2be-4135-a499-c938e99adb61 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:11.775211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628b308-1c39-4b9b-b76a-4a35e06b7305 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:11.776603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628b308-1c39-4b9b-b76a-4a35e06b7305 service placement] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:11.777283 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddedfc9a-3a40-4509-8439-bcb914117a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:11.777900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddedfc9a-3a40-4509-8439-bcb914117a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:11.778182 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddedfc9a-3a40-4509-8439-bcb914117a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:11.778459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddedfc9a-3a40-4509-8439-bcb914117a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:11.779088 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddedfc9a-3a40-4509-8439-bcb914117a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:11.780657 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddedfc9a-3a40-4509-8439-bcb914117a81 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:11.786737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1355/2724] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:11.795873 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1370/2725] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:11.797036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628b308-1c39-4b9b-b76a-4a35e06b7305 service placement] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:11.809553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628b308-1c39-4b9b-b76a-4a35e06b7305 service placement] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:11.814282 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32f04a-daf8-4570-bc90-59fb459a4a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:11.814927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32f04a-daf8-4570-bc90-59fb459a4a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:11.815206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32f04a-daf8-4570-bc90-59fb459a4a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:11.815477 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32f04a-daf8-4570-bc90-59fb459a4a13 admin admin] PATH_INFO: `/v3/projects/9d07bb49295a47a99fd2c89a3873bff1/users/4f4242cefca04a9f81dad6daf175fe68/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:11.831688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32f04a-daf8-4570-bc90-59fb459a4a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d07bb49295a47a99fd2c89a3873bff1, user_id=4f4242cefca04a9f81dad6daf175fe68, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:11.834410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32f04a-daf8-4570-bc90-59fb459a4a13 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:11.912310 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1356/2726] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:57:11 2022] PUT /identity/v3/projects/9d07bb49295a47a99fd2c89a3873bff1/users/4f4242cefca04a9f81dad6daf175fe68/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:11.933565 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db5ac9-8dfb-4a46-ad48-a6714e14a5e1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:11.933894 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db5ac9-8dfb-4a46-ad48-a6714e14a5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:11.934206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db5ac9-8dfb-4a46-ad48-a6714e14a5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:11.962707 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-01db5ac9-8dfb-4a46-ad48-a6714e14a5e1 None None] MFA Rules not processed for user `4f4242cefca04a9f81dad6daf175fe68`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:11.963922 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-01db5ac9-8dfb-4a46-ad48-a6714e14a5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:12.030220 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1371/2727] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:12.042839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32f04a-daf8-4570-bc90-59fb459a4a13 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:12.044789 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-bb32f04a-daf8-4570-bc90-59fb459a4a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:12.107627 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32f04a-daf8-4570-bc90-59fb459a4a13 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:12.111516 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68784e24-6075-4dbb-b44a-5d039a9e23aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:12.112161 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68784e24-6075-4dbb-b44a-5d039a9e23aa service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:12.112435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68784e24-6075-4dbb-b44a-5d039a9e23aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:12.112703 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68784e24-6075-4dbb-b44a-5d039a9e23aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:12.121905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68784e24-6075-4dbb-b44a-5d039a9e23aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:12.123368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68784e24-6075-4dbb-b44a-5d039a9e23aa service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:12.132133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1357/2728] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:22.027231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:22.128063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:INSERT=2 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:23.192651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db5ac9-8dfb-4a46-ad48-a6714e14a5e1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:23.258629 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db5ac9-8dfb-4a46-ad48-a6714e14a5e1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:23.263243 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:23.263868 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:23.264151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:23.264427 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] PATH_INFO: `/v3/users/4f4242cefca04a9f81dad6daf175fe68` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:23.265547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f4242cefca04a9f81dad6daf175fe68)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:23.267622 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:23.464739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f4242cefca04a9f81dad6daf175fe68'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:23.477956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f4242cefca04a9f81dad6daf175fe68'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:23.534379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f4242cefca04a9f81dad6daf175fe68'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:23.535013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f4242cefca04a9f81dad6daf175fe68'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:23.543226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f4242cefca04a9f81dad6daf175fe68'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:23.549603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1372/2729] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:57:23 2022] DELETE /identity/v3/users/4f4242cefca04a9f81dad6daf175fe68 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:23.554096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c96f1-028a-4310-a879-baf7e98459c0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:23.554617 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c96f1-028a-4310-a879-baf7e98459c0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:23.554719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c96f1-028a-4310-a879-baf7e98459c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:23.573529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e25c96f1-028a-4310-a879-baf7e98459c0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:23.574800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e25c96f1-028a-4310-a879-baf7e98459c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:23.632349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1358/2730] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:23.641937 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:23.643373 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:23.702025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0438226-1308-4859-9bf7-c5a425a4d70e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:23.705651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0fe7-ce1f-440b-b647-b0bf89f709d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:23.706276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0fe7-ce1f-440b-b647-b0bf89f709d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:23.706556 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0fe7-ce1f-440b-b647-b0bf89f709d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:23.706829 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0fe7-ce1f-440b-b647-b0bf89f709d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:23.716551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0fe7-ce1f-440b-b647-b0bf89f709d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:23.717936 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0fe7-ce1f-440b-b647-b0bf89f709d5 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:23.728130 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1373/2731] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:24.126938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c96f1-028a-4310-a879-baf7e98459c0 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:24.128437 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e25c96f1-028a-4310-a879-baf7e98459c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:24.142203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c96f1-028a-4310-a879-baf7e98459c0 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:24.147150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:24.147778 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:24.148469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:24.148469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] PATH_INFO: `/v3/projects/9d07bb49295a47a99fd2c89a3873bff1` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:24.151002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d07bb49295a47a99fd2c89a3873bff1)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:24.152925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:24.279341 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d07bb49295a47a99fd2c89a3873bff1', 'request_id': 'req-6de988b1-2076-43f2-ba8e-9521826cb462'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:24.346484 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d07bb49295a47a99fd2c89a3873bff1', 'request_id': 'req-6de988b1-2076-43f2-ba8e-9521826cb462'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:24.353788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d07bb49295a47a99fd2c89a3873bff1', 'request_id': 'req-6de988b1-2076-43f2-ba8e-9521826cb462'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:24.366565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] The token cache is being invalidate because project 9d07bb49295a47a99fd2c89a3873bff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:57:24.366806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:24.367474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6de988b1-2076-43f2-ba8e-9521826cb462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:24.381967 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1359/2732] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:57:24 2022] DELETE /identity/v3/projects/9d07bb49295a47a99fd2c89a3873bff1 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:24.382075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c903fc4-83ba-47f3-828d-3d13f4ae9079 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:24.382075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c903fc4-83ba-47f3-828d-3d13f4ae9079 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:24.382075 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c903fc4-83ba-47f3-828d-3d13f4ae9079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:24.395786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0c903fc4-83ba-47f3-828d-3d13f4ae9079 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:24.396987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0c903fc4-83ba-47f3-828d-3d13f4ae9079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:24.414239 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:24.458063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1374/2733] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:24.466865 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adbec7-9f74-42e0-9ca5-b4f063765e6b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:24.467152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adbec7-9f74-42e0-9ca5-b4f063765e6b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:24.467435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adbec7-9f74-42e0-9ca5-b4f063765e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:24.487584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-94adbec7-9f74-42e0-9ca5-b4f063765e6b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:24.488795 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-94adbec7-9f74-42e0-9ca5-b4f063765e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:24.497227 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1360/2734] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:24.501577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c903fc4-83ba-47f3-828d-3d13f4ae9079 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:24.514714 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c903fc4-83ba-47f3-828d-3d13f4ae9079 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:24.520217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91a996-4897-4650-8a2b-4df46b352f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:24.520812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91a996-4897-4650-8a2b-4df46b352f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:24.521091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91a996-4897-4650-8a2b-4df46b352f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:24.521365 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91a996-4897-4650-8a2b-4df46b352f9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:24.522050 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91a996-4897-4650-8a2b-4df46b352f9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:57:24.522353 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91a996-4897-4650-8a2b-4df46b352f9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:24.523953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91a996-4897-4650-8a2b-4df46b352f9e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:24.536381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1375/2735] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:57:24.540863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adbec7-9f74-42e0-9ca5-b4f063765e6b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:24.550156 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adbec7-9f74-42e0-9ca5-b4f063765e6b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:24.554763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896d406-e934-453c-ba16-ac6491ef29b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:24.555883 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d406-e934-453c-ba16-ac6491ef29b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:24.556147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d406-e934-453c-ba16-ac6491ef29b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:24.556427 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d406-e934-453c-ba16-ac6491ef29b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:24.557050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896d406-e934-453c-ba16-ac6491ef29b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:24.558990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896d406-e934-453c-ba16-ac6491ef29b7 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:24.667615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1361/2736] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:57:24 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:24.671993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91a996-4897-4650-8a2b-4df46b352f9e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:24.682624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91a996-4897-4650-8a2b-4df46b352f9e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:24.739131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cee920-0370-4721-b036-24dde3106335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:24.739734 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cee920-0370-4721-b036-24dde3106335 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:24.740014 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cee920-0370-4721-b036-24dde3106335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:24.740289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cee920-0370-4721-b036-24dde3106335 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:24.740926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cee920-0370-4721-b036-24dde3106335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:24.743537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cee920-0370-4721-b036-24dde3106335 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:24.891462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1376/2737] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:57:24 2022] POST /identity/v3/users => generated 288 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:24.895758 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896d406-e934-453c-ba16-ac6491ef29b7 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:24.906932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896d406-e934-453c-ba16-ac6491ef29b7 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:24.912058 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a29b-1ddd-4586-a513-53521f749aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:24.912763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a29b-1ddd-4586-a513-53521f749aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:24.913047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a29b-1ddd-4586-a513-53521f749aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:24.913323 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a29b-1ddd-4586-a513-53521f749aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:24.914014 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a29b-1ddd-4586-a513-53521f749aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:24.916812 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a29b-1ddd-4586-a513-53521f749aaf admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:24.926924 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1362/2738] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:24.931698 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cee920-0370-4721-b036-24dde3106335 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:24.941998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cee920-0370-4721-b036-24dde3106335 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:24.946955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d929979-92d7-4741-ab6e-837283ab8646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:24.947547 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d929979-92d7-4741-ab6e-837283ab8646 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:24.947834 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d929979-92d7-4741-ab6e-837283ab8646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:24.948115 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d929979-92d7-4741-ab6e-837283ab8646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:24.948756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d929979-92d7-4741-ab6e-837283ab8646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:24.950405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d929979-92d7-4741-ab6e-837283ab8646 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:24.961052 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1377/2739] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:24.967343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a29b-1ddd-4586-a513-53521f749aaf admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:24.978519 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a29b-1ddd-4586-a513-53521f749aaf admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:24.983122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2cf3c-8f3b-4d21-9546-584650f17ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:24.983753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2cf3c-8f3b-4d21-9546-584650f17ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:24.984032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2cf3c-8f3b-4d21-9546-584650f17ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:24.984303 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2cf3c-8f3b-4d21-9546-584650f17ba3 admin admin] PATH_INFO: `/v3/projects/1a5fde45c808477eb29fa36d79bb1a81/users/6e4d0b73c846437aac3ba53a57e98c6a/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:25.120382 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2cf3c-8f3b-4d21-9546-584650f17ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5fde45c808477eb29fa36d79bb1a81, user_id=6e4d0b73c846437aac3ba53a57e98c6a, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:25.123054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2cf3c-8f3b-4d21-9546-584650f17ba3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:25.193788 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1363/2740] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:57:24 2022] PUT /identity/v3/projects/1a5fde45c808477eb29fa36d79bb1a81/users/6e4d0b73c846437aac3ba53a57e98c6a/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:25.207788 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67551aa-256b-4d75-a29b-93c194d32383 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:25.208073 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67551aa-256b-4d75-a29b-93c194d32383 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:25.208349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67551aa-256b-4d75-a29b-93c194d32383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:25.238087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d67551aa-256b-4d75-a29b-93c194d32383 None None] MFA Rules not processed for user `6e4d0b73c846437aac3ba53a57e98c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:25.239291 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d67551aa-256b-4d75-a29b-93c194d32383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:25.302645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1378/2741] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:25.313544 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2cf3c-8f3b-4d21-9546-584650f17ba3 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:25.315255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-34d2cf3c-8f3b-4d21-9546-584650f17ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:25.338221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:25.373029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2cf3c-8f3b-4d21-9546-584650f17ba3 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:25.376804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef2401-fe0a-495b-a3ab-314aa19e21f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:25.377402 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef2401-fe0a-495b-a3ab-314aa19e21f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:25.378292 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef2401-fe0a-495b-a3ab-314aa19e21f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:25.378568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef2401-fe0a-495b-a3ab-314aa19e21f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:25.388495 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef2401-fe0a-495b-a3ab-314aa19e21f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:25.389954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef2401-fe0a-495b-a3ab-314aa19e21f2 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:25.400390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1364/2742] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:26.698304 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67551aa-256b-4d75-a29b-93c194d32383 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:26.699703 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d67551aa-256b-4d75-a29b-93c194d32383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:26.719348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:26.763616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67551aa-256b-4d75-a29b-93c194d32383 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:26.768063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:26.768675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:26.768954 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:26.769228 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] PATH_INFO: `/v3/users/b7a68c8547eb4baf9c2d773b5d32dd4a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:26.770358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a68c8547eb4baf9c2d773b5d32dd4a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:26.772261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:26.961144 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a68c8547eb4baf9c2d773b5d32dd4a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:26.973851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a68c8547eb4baf9c2d773b5d32dd4a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:27.078956 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a68c8547eb4baf9c2d773b5d32dd4a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:27.079594 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a68c8547eb4baf9c2d773b5d32dd4a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:27.087036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a68c8547eb4baf9c2d773b5d32dd4a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:27.092930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1379/2743] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:57:26 2022] DELETE /identity/v3/users/b7a68c8547eb4baf9c2d773b5d32dd4a => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:27.097739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44398ad-90e2-4875-9b39-b785f9f4fb31 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:27.098203 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44398ad-90e2-4875-9b39-b785f9f4fb31 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:27.098325 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44398ad-90e2-4875-9b39-b785f9f4fb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:27.117587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e44398ad-90e2-4875-9b39-b785f9f4fb31 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:27.119563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e44398ad-90e2-4875-9b39-b785f9f4fb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:27.184236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1365/2744] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:27.194572 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:27.196082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:27.249331 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cf0d2-f665-4b80-87a9-14cc952593dc None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:27.249616 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cf0d2-f665-4b80-87a9-14cc952593dc None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:27.250257 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cf0d2-f665-4b80-87a9-14cc952593dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:27.265004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963f0f7-e20f-4b3f-9584-c53b86752fe2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:27.268799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783ad70-743d-4b7d-bff3-7264642235d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:27.269401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783ad70-743d-4b7d-bff3-7264642235d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:27.269680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783ad70-743d-4b7d-bff3-7264642235d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:27.269852 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4b7cf0d2-f665-4b80-87a9-14cc952593dc None None] MFA Rules not processed for user `6e4d0b73c846437aac3ba53a57e98c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:27.269984 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783ad70-743d-4b7d-bff3-7264642235d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:27.271063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4b7cf0d2-f665-4b80-87a9-14cc952593dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:27.279180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783ad70-743d-4b7d-bff3-7264642235d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:27.280602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783ad70-743d-4b7d-bff3-7264642235d8 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:27.290312 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1380/2745] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:27.330150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1366/2746] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:57:27.776485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783ad70-743d-4b7d-bff3-7264642235d8 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:27.778767 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4783ad70-743d-4b7d-bff3-7264642235d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:27.796402 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783ad70-743d-4b7d-bff3-7264642235d8 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:27.813545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:27.814238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:27.814521 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:27.814792 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] PATH_INFO: `/v3/projects/0f985324471c4f9da153fc4b5ae5c0e2` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:27.815913 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f985324471c4f9da153fc4b5ae5c0e2)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:27.817844 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:27.912278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:27.923964 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f985324471c4f9da153fc4b5ae5c0e2', 'request_id': 'req-5aa6403c-8657-4a12-bce0-a91efa32cce1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:27.934285 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f985324471c4f9da153fc4b5ae5c0e2', 'request_id': 'req-5aa6403c-8657-4a12-bce0-a91efa32cce1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:28.032328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f985324471c4f9da153fc4b5ae5c0e2', 'request_id': 'req-5aa6403c-8657-4a12-bce0-a91efa32cce1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:28.045460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] The token cache is being invalidate because project 0f985324471c4f9da153fc4b5ae5c0e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:57:28.045871 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:28.046600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5aa6403c-8657-4a12-bce0-a91efa32cce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:28.048170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1381/2747] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:57:27 2022] DELETE /identity/v3/projects/0f985324471c4f9da153fc4b5ae5c0e2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:28.053328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3ab75-cc4b-454d-86f5-901e95685ae0 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:28.053619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3ab75-cc4b-454d-86f5-901e95685ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:28.053942 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3ab75-cc4b-454d-86f5-901e95685ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:28.074379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6ef3ab75-cc4b-454d-86f5-901e95685ae0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:28.075633 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6ef3ab75-cc4b-454d-86f5-901e95685ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:28.140360 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1367/2748] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:28.148400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab6fc9-16a9-4278-8525-c9c4d8589b27 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:28.148687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab6fc9-16a9-4278-8525-c9c4d8589b27 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:28.149518 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab6fc9-16a9-4278-8525-c9c4d8589b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:28.169896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0aab6fc9-16a9-4278-8525-c9c4d8589b27 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:28.171083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0aab6fc9-16a9-4278-8525-c9c4d8589b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:28.179085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1382/2749] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:28.184081 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3ab75-cc4b-454d-86f5-901e95685ae0 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:28.193901 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3ab75-cc4b-454d-86f5-901e95685ae0 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:28.199224 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871de50-a85f-41b3-83b8-ddb50f1a0894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:28.201255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871de50-a85f-41b3-83b8-ddb50f1a0894 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:28.201255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871de50-a85f-41b3-83b8-ddb50f1a0894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:28.201255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871de50-a85f-41b3-83b8-ddb50f1a0894 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:28.201255 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871de50-a85f-41b3-83b8-ddb50f1a0894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:57:28.201603 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871de50-a85f-41b3-83b8-ddb50f1a0894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:28.202918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871de50-a85f-41b3-83b8-ddb50f1a0894 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:28.205287 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=100 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:28.215049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1368/2750] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:57:28.219340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab6fc9-16a9-4278-8525-c9c4d8589b27 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:28.228755 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab6fc9-16a9-4278-8525-c9c4d8589b27 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:28.233422 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074bc3f-0cb1-465e-8aa3-4c8c29ec5d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:28.234042 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074bc3f-0cb1-465e-8aa3-4c8c29ec5d0b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:28.234323 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074bc3f-0cb1-465e-8aa3-4c8c29ec5d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:28.234602 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074bc3f-0cb1-465e-8aa3-4c8c29ec5d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:28.235229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074bc3f-0cb1-465e-8aa3-4c8c29ec5d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:28.237105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074bc3f-0cb1-465e-8aa3-4c8c29ec5d0b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:28.353999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1383/2751] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:57:28 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:28.358317 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871de50-a85f-41b3-83b8-ddb50f1a0894 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:28.368597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871de50-a85f-41b3-83b8-ddb50f1a0894 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:28.425560 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872e3be-35ad-4fbb-8032-e147ee131e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:28.426188 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872e3be-35ad-4fbb-8032-e147ee131e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:28.426464 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872e3be-35ad-4fbb-8032-e147ee131e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:28.426732 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872e3be-35ad-4fbb-8032-e147ee131e21 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:28.427357 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872e3be-35ad-4fbb-8032-e147ee131e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:28.429334 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872e3be-35ad-4fbb-8032-e147ee131e21 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:28.524088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1369/2752] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:57:28 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:57:28.534199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074bc3f-0cb1-465e-8aa3-4c8c29ec5d0b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:28.545231 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074bc3f-0cb1-465e-8aa3-4c8c29ec5d0b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:28.549826 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2dcff-6ea9-4726-8351-0453e5d866f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:28.550417 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2dcff-6ea9-4726-8351-0453e5d866f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:28.550700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2dcff-6ea9-4726-8351-0453e5d866f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:28.550972 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2dcff-6ea9-4726-8351-0453e5d866f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:28.551612 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2dcff-6ea9-4726-8351-0453e5d866f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:28.553211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2dcff-6ea9-4726-8351-0453e5d866f6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:28.564582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1384/2753] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:57:28.568630 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872e3be-35ad-4fbb-8032-e147ee131e21 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:28.579535 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872e3be-35ad-4fbb-8032-e147ee131e21 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:28.584396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e09d8-fd05-45fc-b7f9-9379f5b43b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:28.585025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e09d8-fd05-45fc-b7f9-9379f5b43b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:28.589954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e09d8-fd05-45fc-b7f9-9379f5b43b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:28.589954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e09d8-fd05-45fc-b7f9-9379f5b43b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:28.589954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e09d8-fd05-45fc-b7f9-9379f5b43b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:28.589954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e09d8-fd05-45fc-b7f9-9379f5b43b41 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:28.599025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1370/2754] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:57:28.603713 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2dcff-6ea9-4726-8351-0453e5d866f6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:28.613927 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2dcff-6ea9-4726-8351-0453e5d866f6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:28.620145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840278e4-5a7d-471b-bc05-2d247a7ff45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:28.624516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840278e4-5a7d-471b-bc05-2d247a7ff45b admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:28.624516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840278e4-5a7d-471b-bc05-2d247a7ff45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:28.624516 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840278e4-5a7d-471b-bc05-2d247a7ff45b admin admin] PATH_INFO: `/v3/projects/9cb618ef3b9b40bf81b560c943f867fc/users/70d910fe942d4052bafa43f9eb499c51/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:28.636136 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840278e4-5a7d-471b-bc05-2d247a7ff45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb618ef3b9b40bf81b560c943f867fc, user_id=70d910fe942d4052bafa43f9eb499c51, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:28.638825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840278e4-5a7d-471b-bc05-2d247a7ff45b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:28.736591 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1385/2755] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:57:28 2022] PUT /identity/v3/projects/9cb618ef3b9b40bf81b560c943f867fc/users/70d910fe942d4052bafa43f9eb499c51/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:28.741612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20224bdd-3e1b-4e65-a67d-9e499d46c72e None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:28.741919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20224bdd-3e1b-4e65-a67d-9e499d46c72e None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:28.742192 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20224bdd-3e1b-4e65-a67d-9e499d46c72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:28.763019 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-20224bdd-3e1b-4e65-a67d-9e499d46c72e None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:28.764217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-20224bdd-3e1b-4e65-a67d-9e499d46c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:28.825202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1371/2756] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:28.834752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840278e4-5a7d-471b-bc05-2d247a7ff45b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:28.837105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-840278e4-5a7d-471b-bc05-2d247a7ff45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:28.893187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840278e4-5a7d-471b-bc05-2d247a7ff45b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:28.896945 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862439b-ada1-47ca-b49f-55dc56e7a959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:28.897548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862439b-ada1-47ca-b49f-55dc56e7a959 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:28.897862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862439b-ada1-47ca-b49f-55dc56e7a959 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:28.898141 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862439b-ada1-47ca-b49f-55dc56e7a959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:28.909978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862439b-ada1-47ca-b49f-55dc56e7a959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:28.910358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862439b-ada1-47ca-b49f-55dc56e7a959 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:28.921541 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1386/2757] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:29.565047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20224bdd-3e1b-4e65-a67d-9e499d46c72e None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:29.566487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-20224bdd-3e1b-4e65-a67d-9e499d46c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:29.578486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20224bdd-3e1b-4e65-a67d-9e499d46c72e None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:29.582469 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7107180-d1a1-449d-ad1b-e4ca49905c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:29.583075 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107180-d1a1-449d-ad1b-e4ca49905c17 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:29.583352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107180-d1a1-449d-ad1b-e4ca49905c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:29.583741 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107180-d1a1-449d-ad1b-e4ca49905c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:29.585487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e7107180-d1a1-449d-ad1b-e4ca49905c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:29.650916 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7107180-d1a1-449d-ad1b-e4ca49905c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:29.652344 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7107180-d1a1-449d-ad1b-e4ca49905c17 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:29.661863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1372/2758] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:30.479004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e447ea7-fc74-4bbc-a5b9-7fc59683faa3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:30.479004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e447ea7-fc74-4bbc-a5b9-7fc59683faa3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:30.479004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e447ea7-fc74-4bbc-a5b9-7fc59683faa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:30.510564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3e447ea7-fc74-4bbc-a5b9-7fc59683faa3 None None] MFA Rules not processed for user `70d910fe942d4052bafa43f9eb499c51`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:30.511779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3e447ea7-fc74-4bbc-a5b9-7fc59683faa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:30.567435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:30.584689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1387/2759] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:30.596136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7107180-d1a1-449d-ad1b-e4ca49905c17 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:30.597641 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e7107180-d1a1-449d-ad1b-e4ca49905c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:30.612759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:30.658389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7107180-d1a1-449d-ad1b-e4ca49905c17 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:30.662149 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db2bf7-c726-4690-83a2-05cc6ab7f58c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:30.662750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2bf7-c726-4690-83a2-05cc6ab7f58c service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:30.663032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2bf7-c726-4690-83a2-05cc6ab7f58c service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:30.663305 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2bf7-c726-4690-83a2-05cc6ab7f58c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:30.672241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db2bf7-c726-4690-83a2-05cc6ab7f58c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:30.673672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db2bf7-c726-4690-83a2-05cc6ab7f58c service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:30.683468 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1373/2760] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:32.619113 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e640d-f58b-443e-9995-fa2ad39ae6a6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:32.620343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e640d-f58b-443e-9995-fa2ad39ae6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:32.620432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e640d-f58b-443e-9995-fa2ad39ae6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:32.639839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6a0e640d-f58b-443e-9995-fa2ad39ae6a6 None None] MFA Rules not processed for user `70d910fe942d4052bafa43f9eb499c51`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:32.641094 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6a0e640d-f58b-443e-9995-fa2ad39ae6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:32.652253 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1388/2761] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:57:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:39.422764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db2bf7-c726-4690-83a2-05cc6ab7f58c service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:39.436142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db2bf7-c726-4690-83a2-05cc6ab7f58c service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:39.443513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ef07b-c5ce-4b9b-a15a-1feb804f2831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:39.444183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ef07b-c5ce-4b9b-a15a-1feb804f2831 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:39.444466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ef07b-c5ce-4b9b-a15a-1feb804f2831 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:39.444740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ef07b-c5ce-4b9b-a15a-1feb804f2831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:39.446529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e55ef07b-c5ce-4b9b-a15a-1feb804f2831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:39.463430 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ef07b-c5ce-4b9b-a15a-1feb804f2831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:39.465134 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ef07b-c5ce-4b9b-a15a-1feb804f2831 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:39.476785 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1374/2762] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 03:57:40.786306 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e640d-f58b-443e-9995-fa2ad39ae6a6 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:40.796523 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e640d-f58b-443e-9995-fa2ad39ae6a6 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:40.804907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a0a8f-f84d-4b1a-89a9-1e44569f8a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:40.805551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a0a8f-f84d-4b1a-89a9-1e44569f8a52 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:40.805886 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a0a8f-f84d-4b1a-89a9-1e44569f8a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:40.806170 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a0a8f-f84d-4b1a-89a9-1e44569f8a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:40.808597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2e2a0a8f-f84d-4b1a-89a9-1e44569f8a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:40.876850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a0a8f-f84d-4b1a-89a9-1e44569f8a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:40.878328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a0a8f-f84d-4b1a-89a9-1e44569f8a52 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:40.889112 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1389/2763] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:57:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:57:48.834500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ef07b-c5ce-4b9b-a15a-1feb804f2831 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:48.835961 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e55ef07b-c5ce-4b9b-a15a-1feb804f2831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:48.849273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ef07b-c5ce-4b9b-a15a-1feb804f2831 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:48.853524 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:48.854905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:48.855183 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:48.855455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] PATH_INFO: `/v3/users/4a26f3f51f9c4e4aa4b1d7ce8bc041d6` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:48.856588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a26f3f51f9c4e4aa4b1d7ce8bc041d6)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:48.858561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:49.036863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a26f3f51f9c4e4aa4b1d7ce8bc041d6'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:49.130463 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a26f3f51f9c4e4aa4b1d7ce8bc041d6'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:49.131088 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a26f3f51f9c4e4aa4b1d7ce8bc041d6'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:49.139250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a26f3f51f9c4e4aa4b1d7ce8bc041d6'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:49.156581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a26f3f51f9c4e4aa4b1d7ce8bc041d6'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:49.162689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1375/2764] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:57:48 2022] DELETE /identity/v3/users/4a26f3f51f9c4e4aa4b1d7ce8bc041d6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:49.473293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a0a8f-f84d-4b1a-89a9-1e44569f8a52 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:49.485525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a0a8f-f84d-4b1a-89a9-1e44569f8a52 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:49.548091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:49.548726 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:49.549010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:49.549289 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] PATH_INFO: `/v3/projects/c265d90202534462b37c19391538b29a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:49.550428 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c265d90202534462b37c19391538b29a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:49.552359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:49.697197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c265d90202534462b37c19391538b29a', 'request_id': 'req-fa6ca34c-55d0-4509-9e64-f471bbe74392'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:49.704345 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c265d90202534462b37c19391538b29a', 'request_id': 'req-fa6ca34c-55d0-4509-9e64-f471bbe74392'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:49.771593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c265d90202534462b37c19391538b29a', 'request_id': 'req-fa6ca34c-55d0-4509-9e64-f471bbe74392'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:49.782161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] The token cache is being invalidate because project c265d90202534462b37c19391538b29a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:57:49.782452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:49.783198 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:49.784961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1390/2765] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:57:49 2022] DELETE /identity/v3/projects/c265d90202534462b37c19391538b29a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:49.788686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:49.790159 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:49.810578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993815c-77b6-4c9c-93dc-7759df099c39 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:49.885172 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:49.886136 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:49.886550 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:49.886950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] PATH_INFO: `/v3/users/3a91cf98350d446daf3a5ac6c462d5be` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:49.888217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a91cf98350d446daf3a5ac6c462d5be)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:49.890291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:49.907739 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:50.115024 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a91cf98350d446daf3a5ac6c462d5be'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:50.187493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a91cf98350d446daf3a5ac6c462d5be'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:50.188144 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a91cf98350d446daf3a5ac6c462d5be'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:50.196037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a91cf98350d446daf3a5ac6c462d5be'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:50.209290 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-64ce5400-c6cf-4cc4-8af0-4c2a985d3474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a91cf98350d446daf3a5ac6c462d5be'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:50.216750 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1376/2766] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:57:49 2022] DELETE /identity/v3/users/3a91cf98350d446daf3a5ac6c462d5be => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:50.469145 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:50.470662 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:50.485174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ca34c-55d0-4509-9e64-f471bbe74392 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:50.490435 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:50.550934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:50.551494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:50.551771 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:50.552095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] PATH_INFO: `/v3/projects/6d045ef7133244a38cb14ad861dcc8d2` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:50.553230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d045ef7133244a38cb14ad861dcc8d2)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:50.555461 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:50.648769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d045ef7133244a38cb14ad861dcc8d2', 'request_id': 'req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:50.655038 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d045ef7133244a38cb14ad861dcc8d2', 'request_id': 'req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:50.716979 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d045ef7133244a38cb14ad861dcc8d2', 'request_id': 'req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:50.728197 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] The token cache is being invalidate because project 6d045ef7133244a38cb14ad861dcc8d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:57:50.728487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:50.729234 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:57:50.730778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1391/2767] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:57:50 2022] DELETE /identity/v3/projects/6d045ef7133244a38cb14ad861dcc8d2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:57:56.093600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53773542-77c2-4999-85d5-e625f7802fc9 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:56.095676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53773542-77c2-4999-85d5-e625f7802fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:56.095676 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53773542-77c2-4999-85d5-e625f7802fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:56.138797 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-53773542-77c2-4999-85d5-e625f7802fc9 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:56.140282 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-53773542-77c2-4999-85d5-e625f7802fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:56.222554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1377/2768] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:57:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:57:56.367191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:56.368804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:56.441977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904a6d0-ff28-4ffc-a1da-dd631a0a819f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:56.446797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5a590-f29d-4747-be21-776da6825cd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:56.447423 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5a590-f29d-4747-be21-776da6825cd9 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:56.448251 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5a590-f29d-4747-be21-776da6825cd9 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:56.448526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5a590-f29d-4747-be21-776da6825cd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:56.458169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5a590-f29d-4747-be21-776da6825cd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:56.460187 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5a590-f29d-4747-be21-776da6825cd9 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:56.469109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1392/2769] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:57:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 03:57:57.236201 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956dca9-ed50-4573-a941-ef73270f1f55 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:57.236694 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956dca9-ed50-4573-a941-ef73270f1f55 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:57.236818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956dca9-ed50-4573-a941-ef73270f1f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:57.256109 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e956dca9-ed50-4573-a941-ef73270f1f55 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:57.257389 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e956dca9-ed50-4573-a941-ef73270f1f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:57.266429 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1378/2770] 10.222.0.42 () {58 vars in 967 bytes} [Sat Sep 17 03:57:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:57.282333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5a590-f29d-4747-be21-776da6825cd9 service swift] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:57.293998 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5a590-f29d-4747-be21-776da6825cd9 service swift] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:57.299405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b266a91-2a69-4032-aa6c-3753e0732d06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:57.300013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266a91-2a69-4032-aa6c-3753e0732d06 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:57.300297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266a91-2a69-4032-aa6c-3753e0732d06 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:57.300582 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266a91-2a69-4032-aa6c-3753e0732d06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:57.309712 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b266a91-2a69-4032-aa6c-3753e0732d06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:57.311321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b266a91-2a69-4032-aa6c-3753e0732d06 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:57.319535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1393/2771] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:57:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:57:57.895774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f90de-d6b1-4447-b4d4-a424f05739fc None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:57.896050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f90de-d6b1-4447-b4d4-a424f05739fc None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:57.896322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f90de-d6b1-4447-b4d4-a424f05739fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:57.916141 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-3e8f90de-d6b1-4447-b4d4-a424f05739fc None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:57.917957 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-3e8f90de-d6b1-4447-b4d4-a424f05739fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:57.924886 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1379/2772] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Sep 17 03:57:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:57.937444 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b266a91-2a69-4032-aa6c-3753e0732d06 service swift] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:57.948132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b266a91-2a69-4032-aa6c-3753e0732d06 service swift] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:57.953272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df11a9-865b-49b7-96d7-30b531c9ac59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:57.953888 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df11a9-865b-49b7-96d7-30b531c9ac59 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:57.954175 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df11a9-865b-49b7-96d7-30b531c9ac59 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:57.954448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df11a9-865b-49b7-96d7-30b531c9ac59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:57.963135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df11a9-865b-49b7-96d7-30b531c9ac59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:57.965961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df11a9-865b-49b7-96d7-30b531c9ac59 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:57.975419 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1394/2773] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:57:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:57:58.609307 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beeabb-0f06-4c15-be7e-a9a9c275def4 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:58.609863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beeabb-0f06-4c15-be7e-a9a9c275def4 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:58.609863 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beeabb-0f06-4c15-be7e-a9a9c275def4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:58.629597 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-28beeabb-0f06-4c15-be7e-a9a9c275def4 None None] MFA Rules not processed for user `b3803e9daa1848ac9feec1bdef4130de`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:57:58.630823 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-28beeabb-0f06-4c15-be7e-a9a9c275def4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:57:58.638990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1380/2774] 10.222.0.42 () {58 vars in 967 bytes} [Sat Sep 17 03:57:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:57:58.651777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df11a9-865b-49b7-96d7-30b531c9ac59 service swift] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:57:58.657313 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:57:58.662993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df11a9-865b-49b7-96d7-30b531c9ac59 service swift] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:57:58.667930 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfafe6-47fa-4f82-8b0b-80ccfc08fbbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1136770f79a47f6a2af0b0a32259209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:57:58.673319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfafe6-47fa-4f82-8b0b-80ccfc08fbbe service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:57:58.673319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfafe6-47fa-4f82-8b0b-80ccfc08fbbe service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:57:58.673319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfafe6-47fa-4f82-8b0b-80ccfc08fbbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:57:58.676982 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfafe6-47fa-4f82-8b0b-80ccfc08fbbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:57:58.678577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfafe6-47fa-4f82-8b0b-80ccfc08fbbe service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:57:58.687211 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1395/2775] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Sep 17 03:57:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 03:58:01.377986 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beeabb-0f06-4c15-be7e-a9a9c275def4 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:01.379648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-28beeabb-0f06-4c15-be7e-a9a9c275def4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:01.447050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beeabb-0f06-4c15-be7e-a9a9c275def4 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:01.452487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:01.453219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:01.453501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:01.453806 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] PATH_INFO: `/v3/users/6e4d0b73c846437aac3ba53a57e98c6a` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:01.455015 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4d0b73c846437aac3ba53a57e98c6a)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:01.456880 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:01.472168 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:58:01.652447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4d0b73c846437aac3ba53a57e98c6a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:01.711208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4d0b73c846437aac3ba53a57e98c6a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:01.711817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4d0b73c846437aac3ba53a57e98c6a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:01.720288 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4d0b73c846437aac3ba53a57e98c6a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:01.737679 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4d0b73c846437aac3ba53a57e98c6a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:01.743692 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1381/2776] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:58:01 2022] DELETE /identity/v3/users/6e4d0b73c846437aac3ba53a57e98c6a => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:58:01.749366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587eced-85ef-40aa-92df-8e74a833de0c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:01.749863 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587eced-85ef-40aa-92df-8e74a833de0c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:01.750000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587eced-85ef-40aa-92df-8e74a833de0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:01.769127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f587eced-85ef-40aa-92df-8e74a833de0c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:58:01.770348 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f587eced-85ef-40aa-92df-8e74a833de0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:01.833388 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1396/2777] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:58:01.843873 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:01.845295 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:01.904272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1449e-5d4f-4608-af96-6317e9a6ded8 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:01.907790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7f38e-b8b3-48b9-ae92-b8e8e285f292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:01.908379 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7f38e-b8b3-48b9-ae92-b8e8e285f292 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:01.908651 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7f38e-b8b3-48b9-ae92-b8e8e285f292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:01.908918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7f38e-b8b3-48b9-ae92-b8e8e285f292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:01.917518 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7f38e-b8b3-48b9-ae92-b8e8e285f292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:01.919536 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7f38e-b8b3-48b9-ae92-b8e8e285f292 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:01.932978 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1382/2778] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:58:02.190763 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587eced-85ef-40aa-92df-8e74a833de0c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:02.192372 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f587eced-85ef-40aa-92df-8e74a833de0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:02.205663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587eced-85ef-40aa-92df-8e74a833de0c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:02.210577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:02.211198 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:02.211510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:02.213910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] PATH_INFO: `/v3/projects/1a5fde45c808477eb29fa36d79bb1a81` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:02.213910 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5fde45c808477eb29fa36d79bb1a81)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:02.214831 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:02.352481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5fde45c808477eb29fa36d79bb1a81', 'request_id': 'req-6691c9d6-74ee-4a17-80ae-ac4d1197d017'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:02.359454 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5fde45c808477eb29fa36d79bb1a81', 'request_id': 'req-6691c9d6-74ee-4a17-80ae-ac4d1197d017'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:02.454780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5fde45c808477eb29fa36d79bb1a81', 'request_id': 'req-6691c9d6-74ee-4a17-80ae-ac4d1197d017'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:02.467649 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] The token cache is being invalidate because project 1a5fde45c808477eb29fa36d79bb1a81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:58:02.467935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:02.468599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6691c9d6-74ee-4a17-80ae-ac4d1197d017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:02.470135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1397/2779] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:58:02 2022] DELETE /identity/v3/projects/1a5fde45c808477eb29fa36d79bb1a81 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:58:02.475352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e7a47-34a1-4bd2-a833-b1ccdc1c0b88 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:02.481500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e7a47-34a1-4bd2-a833-b1ccdc1c0b88 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:02.481500 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e7a47-34a1-4bd2-a833-b1ccdc1c0b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:02.496083 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ae8e7a47-34a1-4bd2-a833-b1ccdc1c0b88 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:58:02.497902 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ae8e7a47-34a1-4bd2-a833-b1ccdc1c0b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:02.564026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1383/2780] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:58:02.572367 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6f65-3ea4-4d1d-8b10-e384be73b24f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:02.572653 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6f65-3ea4-4d1d-8b10-e384be73b24f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:02.572933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6f65-3ea4-4d1d-8b10-e384be73b24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:02.593663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d4cf6f65-3ea4-4d1d-8b10-e384be73b24f None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:58:02.594904 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d4cf6f65-3ea4-4d1d-8b10-e384be73b24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:02.603343 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1398/2781] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:58:02.608080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e7a47-34a1-4bd2-a833-b1ccdc1c0b88 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:02.618060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e7a47-34a1-4bd2-a833-b1ccdc1c0b88 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:02.623253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b783f2-a6ae-49d7-b715-5e3980828d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:02.623854 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b783f2-a6ae-49d7-b715-5e3980828d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:02.624135 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b783f2-a6ae-49d7-b715-5e3980828d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:02.624408 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b783f2-a6ae-49d7-b715-5e3980828d30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:02.625047 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b783f2-a6ae-49d7-b715-5e3980828d30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 03:58:02.625352 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b783f2-a6ae-49d7-b715-5e3980828d30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:02.626979 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b783f2-a6ae-49d7-b715-5e3980828d30 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:02.638925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1384/2782] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 03:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 03:58:02.643512 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf6f65-3ea4-4d1d-8b10-e384be73b24f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:02.652739 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf6f65-3ea4-4d1d-8b10-e384be73b24f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:02.657450 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f47c3-7dca-4b11-9621-73b2aa3cf208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:02.658082 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f47c3-7dca-4b11-9621-73b2aa3cf208 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:02.658364 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f47c3-7dca-4b11-9621-73b2aa3cf208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:02.658638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f47c3-7dca-4b11-9621-73b2aa3cf208 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:02.659263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f47c3-7dca-4b11-9621-73b2aa3cf208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:02.661143 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f47c3-7dca-4b11-9621-73b2aa3cf208 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:02.744625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1399/2783] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 03:58:02 2022] POST /identity/v3/projects => generated 399 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:58:02.750241 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b783f2-a6ae-49d7-b715-5e3980828d30 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:02.760430 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b783f2-a6ae-49d7-b715-5e3980828d30 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:02.778684 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:58:02.819349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c68c45-61df-4dae-9281-03271945f6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:02.819960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c68c45-61df-4dae-9281-03271945f6b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:02.820239 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c68c45-61df-4dae-9281-03271945f6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:02.820513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c68c45-61df-4dae-9281-03271945f6b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:02.821174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c68c45-61df-4dae-9281-03271945f6b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:02.823124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c68c45-61df-4dae-9281-03271945f6b8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:02.925796 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1385/2784] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 03:58:02 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 03:58:02.930226 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f47c3-7dca-4b11-9621-73b2aa3cf208 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:02.941811 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f47c3-7dca-4b11-9621-73b2aa3cf208 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:02.946975 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a577d-7246-4eed-9a1e-1b8702513605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:02.947592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a577d-7246-4eed-9a1e-1b8702513605 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:02.947876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a577d-7246-4eed-9a1e-1b8702513605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:02.948151 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a577d-7246-4eed-9a1e-1b8702513605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:02.948812 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a577d-7246-4eed-9a1e-1b8702513605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:02.950642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a577d-7246-4eed-9a1e-1b8702513605 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:02.961379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1400/2785] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 03:58:02.965392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c68c45-61df-4dae-9281-03271945f6b8 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:02.976658 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c68c45-61df-4dae-9281-03271945f6b8 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:02.982129 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77bdd1-3975-4071-8a76-39fa38da55da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:02.982784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77bdd1-3975-4071-8a76-39fa38da55da admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:02.983067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77bdd1-3975-4071-8a76-39fa38da55da admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:02.983354 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77bdd1-3975-4071-8a76-39fa38da55da admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:02.983992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77bdd1-3975-4071-8a76-39fa38da55da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:02.986693 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77bdd1-3975-4071-8a76-39fa38da55da admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:02.998186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1386/2786] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 03:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 03:58:03.002308 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a577d-7246-4eed-9a1e-1b8702513605 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:03.013230 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a577d-7246-4eed-9a1e-1b8702513605 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:03.018017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e707a6b-0b89-4cae-adb1-d52389c70515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:03.018752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e707a6b-0b89-4cae-adb1-d52389c70515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:03.019030 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e707a6b-0b89-4cae-adb1-d52389c70515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:03.019302 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e707a6b-0b89-4cae-adb1-d52389c70515 admin admin] PATH_INFO: `/v3/projects/4b9227a32f1b46e8ae3f6c5428abfe08/users/cf2d32e97edb4e35bef0b0cd5a21426c/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:03.034646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e707a6b-0b89-4cae-adb1-d52389c70515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9227a32f1b46e8ae3f6c5428abfe08, user_id=cf2d32e97edb4e35bef0b0cd5a21426c, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:03.037266 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e707a6b-0b89-4cae-adb1-d52389c70515 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:03.126670 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1401/2787] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 03:58:03 2022] PUT /identity/v3/projects/4b9227a32f1b46e8ae3f6c5428abfe08/users/cf2d32e97edb4e35bef0b0cd5a21426c/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:58:03.136939 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623b514-939b-4af9-aae0-5b21b06f0a20 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:03.137219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623b514-939b-4af9-aae0-5b21b06f0a20 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:03.137489 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623b514-939b-4af9-aae0-5b21b06f0a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:03.168309 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6623b514-939b-4af9-aae0-5b21b06f0a20 None None] MFA Rules not processed for user `cf2d32e97edb4e35bef0b0cd5a21426c`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:58:03.170186 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6623b514-939b-4af9-aae0-5b21b06f0a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:03.236702 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1387/2788] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:58:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:58:03.248111 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e707a6b-0b89-4cae-adb1-d52389c70515 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:03.250017 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8e707a6b-0b89-4cae-adb1-d52389c70515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:03.261059 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:58:03.310527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e707a6b-0b89-4cae-adb1-d52389c70515 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:03.314218 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161fa6c-188e-43b4-b8cb-6dba67fd7d51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:03.314825 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161fa6c-188e-43b4-b8cb-6dba67fd7d51 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:03.315104 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161fa6c-188e-43b4-b8cb-6dba67fd7d51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:03.315380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161fa6c-188e-43b4-b8cb-6dba67fd7d51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:03.323993 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161fa6c-188e-43b4-b8cb-6dba67fd7d51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:03.325552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161fa6c-188e-43b4-b8cb-6dba67fd7d51 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:03.334258 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1402/2789] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:58:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:58:13.231563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:58:13.345370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=22 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:58:15.360814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623b514-939b-4af9-aae0-5b21b06f0a20 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:15.422090 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623b514-939b-4af9-aae0-5b21b06f0a20 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:15.426590 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:15.427207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:15.427487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:15.427761 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] PATH_INFO: `/v3/users/cf2d32e97edb4e35bef0b0cd5a21426c` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:15.428870 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf2d32e97edb4e35bef0b0cd5a21426c)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:15.430818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:15.601966 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf2d32e97edb4e35bef0b0cd5a21426c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:15.684612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf2d32e97edb4e35bef0b0cd5a21426c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:15.685228 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf2d32e97edb4e35bef0b0cd5a21426c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:15.693277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf2d32e97edb4e35bef0b0cd5a21426c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:15.705166 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf2d32e97edb4e35bef0b0cd5a21426c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:15.711072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1388/2790] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 03:58:15 2022] DELETE /identity/v3/users/cf2d32e97edb4e35bef0b0cd5a21426c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:58:15.715705 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef0c90-6ff7-4cff-a449-825babaf04f1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:15.716174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef0c90-6ff7-4cff-a449-825babaf04f1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:15.716290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef0c90-6ff7-4cff-a449-825babaf04f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:15.735977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-78ef0c90-6ff7-4cff-a449-825babaf04f1 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:58:15.737171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-78ef0c90-6ff7-4cff-a449-825babaf04f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:15.797400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1403/2791] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:58:15.805655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:15.807454 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:15.866178 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367858a-5b23-4f44-b66f-6df976aaa60f admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:15.869753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57f386-6a8e-41ca-9117-00705b2a5ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:15.871897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f386-6a8e-41ca-9117-00705b2a5ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:15.871897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f386-6a8e-41ca-9117-00705b2a5ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:15.871897 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f386-6a8e-41ca-9117-00705b2a5ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:15.880122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57f386-6a8e-41ca-9117-00705b2a5ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:15.881542 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57f386-6a8e-41ca-9117-00705b2a5ad1 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:15.891242 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1389/2792] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:58:16.263665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef0c90-6ff7-4cff-a449-825babaf04f1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:16.265434 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-78ef0c90-6ff7-4cff-a449-825babaf04f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:16.277677 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef0c90-6ff7-4cff-a449-825babaf04f1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:16.283039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:16.283642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:16.283923 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:16.284199 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] PATH_INFO: `/v3/projects/4b9227a32f1b46e8ae3f6c5428abfe08` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:16.285315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b9227a32f1b46e8ae3f6c5428abfe08)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:16.287264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:16.393577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9227a32f1b46e8ae3f6c5428abfe08', 'request_id': 'req-30b4f8b2-d4e7-471c-a321-2958a798fbb3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:16.399708 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9227a32f1b46e8ae3f6c5428abfe08', 'request_id': 'req-30b4f8b2-d4e7-471c-a321-2958a798fbb3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:16.473272 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9227a32f1b46e8ae3f6c5428abfe08', 'request_id': 'req-30b4f8b2-d4e7-471c-a321-2958a798fbb3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:16.484779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] The token cache is being invalidate because project 4b9227a32f1b46e8ae3f6c5428abfe08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 03:58:16.485058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:16.485675 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 03:58:16.487195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1404/2793] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 03:58:16 2022] DELETE /identity/v3/projects/4b9227a32f1b46e8ae3f6c5428abfe08 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 03:58:25.886800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:58:26.481105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:58:39.691451 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57f386-6a8e-41ca-9117-00705b2a5ad1 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:39.693400 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-8e57f386-6a8e-41ca-9117-00705b2a5ad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:39.753563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57f386-6a8e-41ca-9117-00705b2a5ad1 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:39.804181 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a2ef0-7971-4ddc-9ef1-7acd58560ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:39.805422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2ef0-7971-4ddc-9ef1-7acd58560ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:39.805422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2ef0-7971-4ddc-9ef1-7acd58560ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:39.805619 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2ef0-7971-4ddc-9ef1-7acd58560ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:39.807277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-916a2ef0-7971-4ddc-9ef1-7acd58560ec8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:39.877061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a2ef0-7971-4ddc-9ef1-7acd58560ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:39.879414 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a2ef0-7971-4ddc-9ef1-7acd58560ec8 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:39.890225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1390/2794] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:58:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:58:43.902032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:58:43.903464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:43.959978 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4f8b2-d4e7-471c-a321-2958a798fbb3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:58:43.963514 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc1511-9902-4859-a412-e1d5f3d053d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:58:43.964110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc1511-9902-4859-a412-e1d5f3d053d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:43.964386 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc1511-9902-4859-a412-e1d5f3d053d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:43.964654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc1511-9902-4859-a412-e1d5f3d053d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:43.966371 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bddc1511-9902-4859-a412-e1d5f3d053d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:43.980355 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc1511-9902-4859-a412-e1d5f3d053d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:58:43.984907 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc1511-9902-4859-a412-e1d5f3d053d7 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:58:43.993528 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1405/2795] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:58:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:58:49.885372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=74 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:58:53.988537 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=26 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:58:57.940291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaba6f-1b6d-4646-a685-c4e3350d6950 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:57.940672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaba6f-1b6d-4646-a685-c4e3350d6950 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:57.940672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaba6f-1b6d-4646-a685-c4e3350d6950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:57.960265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-7fcaba6f-1b6d-4646-a685-c4e3350d6950 None None] MFA Rules not processed for user `3f46252462244a0a84180c4726d15f94`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:58:57.962149 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7fcaba6f-1b6d-4646-a685-c4e3350d6950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:57.977201 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421342b-87b2-44a3-8a15-47f4b5aa6a27 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:58:57.977823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421342b-87b2-44a3-8a15-47f4b5aa6a27 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:58:57.977823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421342b-87b2-44a3-8a15-47f4b5aa6a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:58:57.998525 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a421342b-87b2-44a3-8a15-47f4b5aa6a27 None None] MFA Rules not processed for user `70d910fe942d4052bafa43f9eb499c51`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 03:58:57.999717 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a421342b-87b2-44a3-8a15-47f4b5aa6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:58:58.022834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1391/2796] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:58:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 03:58:58.063005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1406/2797] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 03:58:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 03:59:08.015960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=25 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:59:08.057498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=25 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:59:39.051375 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcaba6f-1b6d-4646-a685-c4e3350d6950 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:59:39.052833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7fcaba6f-1b6d-4646-a685-c4e3350d6950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:59:39.074219 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcaba6f-1b6d-4646-a685-c4e3350d6950 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:59:39.096174 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242744c-73f1-420c-8d03-0cf1ab59d588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:59:39.097069 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242744c-73f1-420c-8d03-0cf1ab59d588 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:59:39.097472 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242744c-73f1-420c-8d03-0cf1ab59d588 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:59:39.098196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242744c-73f1-420c-8d03-0cf1ab59d588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:59:39.100223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7242744c-73f1-420c-8d03-0cf1ab59d588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:59:39.114085 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242744c-73f1-420c-8d03-0cf1ab59d588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:59:39.116480 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242744c-73f1-420c-8d03-0cf1ab59d588 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:59:39.128055 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1392/2798] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:59:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:59:39.535572 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421342b-87b2-44a3-8a15-47f4b5aa6a27 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:59:39.537098 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a421342b-87b2-44a3-8a15-47f4b5aa6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:59:39.550150 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421342b-87b2-44a3-8a15-47f4b5aa6a27 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:59:39.553953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722c344-3a23-45a1-8fff-ffe3ea5fcd8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:59:39.554554 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722c344-3a23-45a1-8fff-ffe3ea5fcd8f service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:59:39.554830 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722c344-3a23-45a1-8fff-ffe3ea5fcd8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:59:39.555099 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722c344-3a23-45a1-8fff-ffe3ea5fcd8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:59:39.556777 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a722c344-3a23-45a1-8fff-ffe3ea5fcd8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:59:39.567134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722c344-3a23-45a1-8fff-ffe3ea5fcd8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:59:39.568543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722c344-3a23-45a1-8fff-ffe3ea5fcd8f service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:59:39.576967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1407/2799] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:59:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:59:41.981250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242744c-73f1-420c-8d03-0cf1ab59d588 service nova] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 03:59:41.982808 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7242744c-73f1-420c-8d03-0cf1ab59d588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:59:41.998026 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242744c-73f1-420c-8d03-0cf1ab59d588 service nova] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 03:59:42.004063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d3621-c178-4960-ad35-d5a26ed1a8be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 03:59:42.004681 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d3621-c178-4960-ad35-d5a26ed1a8be service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 03:59:42.004962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d3621-c178-4960-ad35-d5a26ed1a8be service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 03:59:42.005236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d3621-c178-4960-ad35-d5a26ed1a8be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 03:59:42.007051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-667d3621-c178-4960-ad35-d5a26ed1a8be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 03:59:42.021154 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d3621-c178-4960-ad35-d5a26ed1a8be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 03:59:42.022800 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d3621-c178-4960-ad35-d5a26ed1a8be service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 03:59:42.035646 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1393/2800] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 03:59:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 03:59:49.572926 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 03:59:52.029566 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=12 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:00.236349 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722c344-3a23-45a1-8fff-ffe3ea5fcd8f service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:00.245124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722c344-3a23-45a1-8fff-ffe3ea5fcd8f service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:00.249501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d29f-781e-4b87-989d-0bfc84afed2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:00.250161 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d29f-781e-4b87-989d-0bfc84afed2d service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:00.250491 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d29f-781e-4b87-989d-0bfc84afed2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:00.250765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d29f-781e-4b87-989d-0bfc84afed2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:00.252480 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c2f2d29f-781e-4b87-989d-0bfc84afed2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:00.316616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2d29f-781e-4b87-989d-0bfc84afed2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:00.318096 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2d29f-781e-4b87-989d-0bfc84afed2d service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:00.327918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1408/2801] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:00:00.432975 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d3621-c178-4960-ad35-d5a26ed1a8be service nova] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:00.446615 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d3621-c178-4960-ad35-d5a26ed1a8be service nova] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:00.452390 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcabb8-2037-4487-8f2a-957fc056ccd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9901608b62344115aa561d00bfe88c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:00.453025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcabb8-2037-4487-8f2a-957fc056ccd8 service nova] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:00.453299 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcabb8-2037-4487-8f2a-957fc056ccd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:00.453563 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcabb8-2037-4487-8f2a-957fc056ccd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:00.455331 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e2bcabb8-2037-4487-8f2a-957fc056ccd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:00.467815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcabb8-2037-4487-8f2a-957fc056ccd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:00.469421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcabb8-2037-4487-8f2a-957fc056ccd8 service nova] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:00.479919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1394/2802] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:00:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 04:00:07.937190 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d29f-781e-4b87-989d-0bfc84afed2d service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:07.938704 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c2f2d29f-781e-4b87-989d-0bfc84afed2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:07.951055 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d29f-781e-4b87-989d-0bfc84afed2d service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:07.958493 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:07.959118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:07.959394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:07.959663 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] PATH_INFO: `/v3/users/89faad7b1c26412cac5c2c624fa3f0ed` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:07.960780 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=89faad7b1c26412cac5c2c624fa3f0ed)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:07.962861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:08.166694 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89faad7b1c26412cac5c2c624fa3f0ed'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:08.178680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89faad7b1c26412cac5c2c624fa3f0ed'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:08.266604 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89faad7b1c26412cac5c2c624fa3f0ed'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:08.267439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89faad7b1c26412cac5c2c624fa3f0ed'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:08.275039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89faad7b1c26412cac5c2c624fa3f0ed'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:08.281100 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1409/2803] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:00:07 2022] DELETE /identity/v3/users/89faad7b1c26412cac5c2c624fa3f0ed => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:08.533561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcabb8-2037-4487-8f2a-957fc056ccd8 service nova] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:08.545621 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcabb8-2037-4487-8f2a-957fc056ccd8 service nova] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:08.602329 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:08.602862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:08.603139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:08.603412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] PATH_INFO: `/v3/projects/cb2d22c677bb48a1aa50631997fc0741` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:08.604582 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb2d22c677bb48a1aa50631997fc0741)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:08.607122 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:08.722245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb2d22c677bb48a1aa50631997fc0741', 'request_id': 'req-4cc1906d-b316-4e3b-9663-4b940b6af451'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:08.783111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb2d22c677bb48a1aa50631997fc0741', 'request_id': 'req-4cc1906d-b316-4e3b-9663-4b940b6af451'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:08.789002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb2d22c677bb48a1aa50631997fc0741', 'request_id': 'req-4cc1906d-b316-4e3b-9663-4b940b6af451'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:08.801078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] The token cache is being invalidate because project cb2d22c677bb48a1aa50631997fc0741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:00:08.801372 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:08.802079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:08.803994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1395/2804] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:00:08 2022] DELETE /identity/v3/projects/cb2d22c677bb48a1aa50631997fc0741 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:08.807797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:08.809866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:08.823164 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c177bdf-d144-4d6b-a260-a15c50e8a6bf admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:08.880807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:08.881358 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:08.881642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:08.881951 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] PATH_INFO: `/v3/users/5a9252ac49fc4c79b7bd5e6fc425e1a2` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:08.883085 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a9252ac49fc4c79b7bd5e6fc425e1a2)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:08.885002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:08.890481 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:09.056118 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a9252ac49fc4c79b7bd5e6fc425e1a2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:09.069889 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a9252ac49fc4c79b7bd5e6fc425e1a2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:09.147861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a9252ac49fc4c79b7bd5e6fc425e1a2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:09.148433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a9252ac49fc4c79b7bd5e6fc425e1a2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:09.157095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a9252ac49fc4c79b7bd5e6fc425e1a2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:09.163292 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1410/2805] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:00:08 2022] DELETE /identity/v3/users/5a9252ac49fc4c79b7bd5e6fc425e1a2 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:09.419905 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:09.423956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:09.435377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1906d-b316-4e3b-9663-4b940b6af451 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:09.502890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:09.503513 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:09.503791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:09.504061 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] PATH_INFO: `/v3/projects/ac0ebe0ca1bd4032b42483825e838762` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:09.505164 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac0ebe0ca1bd4032b42483825e838762)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:09.507096 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:09.528297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:09.622660 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac0ebe0ca1bd4032b42483825e838762', 'request_id': 'req-56a6b3e4-4d40-4fb2-af51-09c208f8e123'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:09.718148 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac0ebe0ca1bd4032b42483825e838762', 'request_id': 'req-56a6b3e4-4d40-4fb2-af51-09c208f8e123'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:09.724760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac0ebe0ca1bd4032b42483825e838762', 'request_id': 'req-56a6b3e4-4d40-4fb2-af51-09c208f8e123'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:09.738321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] The token cache is being invalidate because project ac0ebe0ca1bd4032b42483825e838762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:00:09.738321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:09.738321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:09.749118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1396/2806] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:00:09 2022] DELETE /identity/v3/projects/ac0ebe0ca1bd4032b42483825e838762 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:10.732905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:10.735033 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:10.753645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c86081-712f-4e6a-bc01-97862672a461 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:10.842242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:10.842882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:10.843257 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:10.843535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] PATH_INFO: `/v3/users/70d910fe942d4052bafa43f9eb499c51` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:10.844772 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d910fe942d4052bafa43f9eb499c51)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:10.847039 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:11.056528 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d910fe942d4052bafa43f9eb499c51'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:11.071379 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d910fe942d4052bafa43f9eb499c51'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:11.161850 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d910fe942d4052bafa43f9eb499c51'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:11.162477 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d910fe942d4052bafa43f9eb499c51'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:11.171645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6ad6016b-159c-47c5-a93b-f775c829609c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d910fe942d4052bafa43f9eb499c51'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:11.177793 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1411/2807] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:00:10 2022] DELETE /identity/v3/users/70d910fe942d4052bafa43f9eb499c51 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:11.545249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:11.547250 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:11.562059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6b3e4-4d40-4fb2-af51-09c208f8e123 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:11.633475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:11.634345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:11.634512 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:11.634666 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] PATH_INFO: `/v3/projects/9cb618ef3b9b40bf81b560c943f867fc` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:11.635896 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb618ef3b9b40bf81b560c943f867fc)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:11.637862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:11.755392 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb618ef3b9b40bf81b560c943f867fc', 'request_id': 'req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:11.834350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb618ef3b9b40bf81b560c943f867fc', 'request_id': 'req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:11.841776 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb618ef3b9b40bf81b560c943f867fc', 'request_id': 'req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:11.853425 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] The token cache is being invalidate because project 9cb618ef3b9b40bf81b560c943f867fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:00:11.853712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:11.854367 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-d58c82eb-e1df-4cb9-9c8b-47e7372a17e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:11.856974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1397/2808] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:00:11 2022] DELETE /identity/v3/projects/9cb618ef3b9b40bf81b560c943f867fc => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:11.861900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34affbbf-3340-4d7b-841b-1e6bab9b56cd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:11.862447 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34affbbf-3340-4d7b-841b-1e6bab9b56cd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:11.862599 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34affbbf-3340-4d7b-841b-1e6bab9b56cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:11.870963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:11.882418 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-34affbbf-3340-4d7b-841b-1e6bab9b56cd None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:00:11.883633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-34affbbf-3340-4d7b-841b-1e6bab9b56cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:11.944013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1412/2809] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:00:11.951830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0388-52e0-492c-94e7-d3d26c4e4f8d None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:11.952111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0388-52e0-492c-94e7-d3d26c4e4f8d None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:11.952385 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0388-52e0-492c-94e7-d3d26c4e4f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:11.973749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-e18c0388-52e0-492c-94e7-d3d26c4e4f8d None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:00:11.974953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-e18c0388-52e0-492c-94e7-d3d26c4e4f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:11.983478 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1398/2810] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:00:11.987639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34affbbf-3340-4d7b-841b-1e6bab9b56cd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:11.997254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34affbbf-3340-4d7b-841b-1e6bab9b56cd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:12.001860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a18dc-65b3-487a-868a-29eeb4100e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:12.002479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a18dc-65b3-487a-868a-29eeb4100e40 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:12.002754 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a18dc-65b3-487a-868a-29eeb4100e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:12.003024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a18dc-65b3-487a-868a-29eeb4100e40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:12.003656 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a18dc-65b3-487a-868a-29eeb4100e40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:00:12.003953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a18dc-65b3-487a-868a-29eeb4100e40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:12.005532 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a18dc-65b3-487a-868a-29eeb4100e40 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:12.016955 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1413/2811] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 04:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:00:12.021272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0388-52e0-492c-94e7-d3d26c4e4f8d None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:12.031213 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0388-52e0-492c-94e7-d3d26c4e4f8d None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:12.035952 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e2d4d-e630-4b7e-9a19-3b32667ad6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:12.036539 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e2d4d-e630-4b7e-9a19-3b32667ad6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:12.036817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e2d4d-e630-4b7e-9a19-3b32667ad6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:12.037089 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e2d4d-e630-4b7e-9a19-3b32667ad6b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:12.037705 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e2d4d-e630-4b7e-9a19-3b32667ad6b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:12.039700 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e2d4d-e630-4b7e-9a19-3b32667ad6b4 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:12.115960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:12.122932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1399/2812] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 04:00:12 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:00:12.127179 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a18dc-65b3-487a-868a-29eeb4100e40 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:12.138135 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a18dc-65b3-487a-868a-29eeb4100e40 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:12.195457 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c213e9-182c-4935-b4db-1a35c2b571ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:12.196087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c213e9-182c-4935-b4db-1a35c2b571ca admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:12.196368 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c213e9-182c-4935-b4db-1a35c2b571ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:12.196642 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c213e9-182c-4935-b4db-1a35c2b571ca admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:12.197278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c213e9-182c-4935-b4db-1a35c2b571ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:12.199324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c213e9-182c-4935-b4db-1a35c2b571ca admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:12.267108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1414/2813] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 04:00:12 2022] POST /identity/v3/users => generated 295 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:00:12.271197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e2d4d-e630-4b7e-9a19-3b32667ad6b4 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:12.281573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e2d4d-e630-4b7e-9a19-3b32667ad6b4 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:12.286620 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab836118-1d3f-4bed-b8f5-48906ae3b0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:12.287209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab836118-1d3f-4bed-b8f5-48906ae3b0cd admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:12.287486 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab836118-1d3f-4bed-b8f5-48906ae3b0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:12.287752 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab836118-1d3f-4bed-b8f5-48906ae3b0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:12.288421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab836118-1d3f-4bed-b8f5-48906ae3b0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:12.290107 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab836118-1d3f-4bed-b8f5-48906ae3b0cd admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:12.300504 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1400/2814] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:00:12.307303 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c213e9-182c-4935-b4db-1a35c2b571ca admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:12.318408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c213e9-182c-4935-b4db-1a35c2b571ca admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:12.323472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a123414-421e-418f-8c7c-6922f58c7755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:12.324077 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a123414-421e-418f-8c7c-6922f58c7755 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:12.324392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a123414-421e-418f-8c7c-6922f58c7755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:12.324731 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a123414-421e-418f-8c7c-6922f58c7755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:12.325377 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a123414-421e-418f-8c7c-6922f58c7755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:12.327064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a123414-421e-418f-8c7c-6922f58c7755 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:12.339503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1415/2815] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:00:12.347552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab836118-1d3f-4bed-b8f5-48906ae3b0cd admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:12.358659 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab836118-1d3f-4bed-b8f5-48906ae3b0cd admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:12.363293 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8223fd-1c58-49ad-ab59-7b05506b54f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:12.363920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8223fd-1c58-49ad-ab59-7b05506b54f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:12.364193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8223fd-1c58-49ad-ab59-7b05506b54f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:12.364466 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8223fd-1c58-49ad-ab59-7b05506b54f9 admin admin] PATH_INFO: `/v3/projects/6c0aa058726e4aa89448d1f273bc479a/users/048e949ea5334e9ba9b5fbf0c2fc3544/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:12.379991 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8223fd-1c58-49ad-ab59-7b05506b54f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0aa058726e4aa89448d1f273bc479a, user_id=048e949ea5334e9ba9b5fbf0c2fc3544, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:12.382591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8223fd-1c58-49ad-ab59-7b05506b54f9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:12.461155 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1401/2816] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:00:12 2022] PUT /identity/v3/projects/6c0aa058726e4aa89448d1f273bc479a/users/048e949ea5334e9ba9b5fbf0c2fc3544/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:12.473680 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37a17d-4353-46c4-8d57-7ecfd8c08d42 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:12.473995 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37a17d-4353-46c4-8d57-7ecfd8c08d42 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:12.474269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37a17d-4353-46c4-8d57-7ecfd8c08d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:12.502322 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8b37a17d-4353-46c4-8d57-7ecfd8c08d42 None None] MFA Rules not processed for user `048e949ea5334e9ba9b5fbf0c2fc3544`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:00:12.503519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8b37a17d-4353-46c4-8d57-7ecfd8c08d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:12.562445 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:12.565929 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1416/2817] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:00:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:00:12.576982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8223fd-1c58-49ad-ab59-7b05506b54f9 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:12.578685 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-5f8223fd-1c58-49ad-ab59-7b05506b54f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:12.641338 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8223fd-1c58-49ad-ab59-7b05506b54f9 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:12.645024 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e19ac-1910-4e09-8cc1-9ebc483d3693 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:12.645622 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e19ac-1910-4e09-8cc1-9ebc483d3693 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:12.645927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e19ac-1910-4e09-8cc1-9ebc483d3693 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:12.646197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e19ac-1910-4e09-8cc1-9ebc483d3693 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:12.660271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e19ac-1910-4e09-8cc1-9ebc483d3693 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:12.660271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e19ac-1910-4e09-8cc1-9ebc483d3693 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:12.665765 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1402/2818] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:00:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:00:22.661746 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:32.518720 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37a17d-4353-46c4-8d57-7ecfd8c08d42 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:32.581963 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37a17d-4353-46c4-8d57-7ecfd8c08d42 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:32.586407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:32.587035 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:32.587311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:32.587581 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] PATH_INFO: `/v3/users/048e949ea5334e9ba9b5fbf0c2fc3544` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:32.588689 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=048e949ea5334e9ba9b5fbf0c2fc3544)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:32.590687 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:32.748535 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '048e949ea5334e9ba9b5fbf0c2fc3544'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:32.761255 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '048e949ea5334e9ba9b5fbf0c2fc3544'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:32.832765 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '048e949ea5334e9ba9b5fbf0c2fc3544'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:32.833390 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '048e949ea5334e9ba9b5fbf0c2fc3544'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:32.841319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '048e949ea5334e9ba9b5fbf0c2fc3544'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:32.847387 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1417/2819] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:00:32 2022] DELETE /identity/v3/users/048e949ea5334e9ba9b5fbf0c2fc3544 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:32.852121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f5a3-1c60-417f-9a73-511a090571b2 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:32.852589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f5a3-1c60-417f-9a73-511a090571b2 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:32.853586 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f5a3-1c60-417f-9a73-511a090571b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:32.871520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-4782f5a3-1c60-417f-9a73-511a090571b2 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:00:32.872704 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4782f5a3-1c60-417f-9a73-511a090571b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:32.930826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1403/2820] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:00:32.940724 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:32.942156 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:32.999415 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeb887-f89e-4282-b2fa-0c3f940e98c1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:33.003370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92a9c3-ef7f-4786-aec3-ed0b9f6af291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:33.003967 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92a9c3-ef7f-4786-aec3-ed0b9f6af291 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:33.004242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92a9c3-ef7f-4786-aec3-ed0b9f6af291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:33.004511 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92a9c3-ef7f-4786-aec3-ed0b9f6af291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:33.013669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92a9c3-ef7f-4786-aec3-ed0b9f6af291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:33.015107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92a9c3-ef7f-4786-aec3-ed0b9f6af291 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:33.025028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1418/2821] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:00:33.411827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f5a3-1c60-417f-9a73-511a090571b2 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:33.413268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4782f5a3-1c60-417f-9a73-511a090571b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:33.426125 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f5a3-1c60-417f-9a73-511a090571b2 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:33.430699 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:33.431321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:33.431690 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:33.431960 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] PATH_INFO: `/v3/projects/6c0aa058726e4aa89448d1f273bc479a` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:33.433103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0aa058726e4aa89448d1f273bc479a)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:33.435021 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:33.546826 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0aa058726e4aa89448d1f273bc479a', 'request_id': 'req-90a6f6b1-483e-403f-8298-06163d09b288'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:33.632600 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0aa058726e4aa89448d1f273bc479a', 'request_id': 'req-90a6f6b1-483e-403f-8298-06163d09b288'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:33.638655 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0aa058726e4aa89448d1f273bc479a', 'request_id': 'req-90a6f6b1-483e-403f-8298-06163d09b288'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:33.651074 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] The token cache is being invalidate because project 6c0aa058726e4aa89448d1f273bc479a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:00:33.654318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:33.654318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:33.654318 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1404/2822] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:00:33 2022] DELETE /identity/v3/projects/6c0aa058726e4aa89448d1f273bc479a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:40.841809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92a9c3-ef7f-4786-aec3-ed0b9f6af291 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:40.843350 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bf92a9c3-ef7f-4786-aec3-ed0b9f6af291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:40.855536 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92a9c3-ef7f-4786-aec3-ed0b9f6af291 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:40.873458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:40.903513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fb1bd-5b75-43fe-965c-16b09ce39a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:40.904105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb1bd-5b75-43fe-965c-16b09ce39a9b service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:40.904382 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb1bd-5b75-43fe-965c-16b09ce39a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:40.904651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb1bd-5b75-43fe-965c-16b09ce39a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:40.906520 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-938fb1bd-5b75-43fe-965c-16b09ce39a9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:40.969683 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fb1bd-5b75-43fe-965c-16b09ce39a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:40.971057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fb1bd-5b75-43fe-965c-16b09ce39a9b service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:40.981790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1419/2823] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:00:43.647791 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:44.714771 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:44.716216 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:44.730569 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6f6b1-483e-403f-8298-06163d09b288 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:44.735165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:44.735786 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:44.736067 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:44.736340 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] PATH_INFO: `/v3/users/3f46252462244a0a84180c4726d15f94` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:44.737596 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f46252462244a0a84180c4726d15f94)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:44.739561 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:44.882271 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f46252462244a0a84180c4726d15f94'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:44.930377 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f46252462244a0a84180c4726d15f94'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:44.930982 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f46252462244a0a84180c4726d15f94'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:44.937956 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f46252462244a0a84180c4726d15f94'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:44.950717 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-45669492-0240-4fd1-ba7d-78a677328511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f46252462244a0a84180c4726d15f94'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:44.956632 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1405/2824] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:00:44 2022] DELETE /identity/v3/users/3f46252462244a0a84180c4726d15f94 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:45.184722 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fb1bd-5b75-43fe-965c-16b09ce39a9b service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:45.195965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fb1bd-5b75-43fe-965c-16b09ce39a9b service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:45.253125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:45.253756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:45.254047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:45.254977 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] PATH_INFO: `/v3/projects/ddf27ec567494d03bd1d2469aef97264` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:45.256105 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddf27ec567494d03bd1d2469aef97264)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:45.258065 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:45.370527 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddf27ec567494d03bd1d2469aef97264', 'request_id': 'req-546c2603-6f41-4ce8-a358-ed22d9e8ddad'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:45.377087 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddf27ec567494d03bd1d2469aef97264', 'request_id': 'req-546c2603-6f41-4ce8-a358-ed22d9e8ddad'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:45.443799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddf27ec567494d03bd1d2469aef97264', 'request_id': 'req-546c2603-6f41-4ce8-a358-ed22d9e8ddad'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:45.454548 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] The token cache is being invalidate because project ddf27ec567494d03bd1d2469aef97264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:00:45.454837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:45.455458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-546c2603-6f41-4ce8-a358-ed22d9e8ddad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:00:45.456947 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1420/2825] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:00:45 2022] DELETE /identity/v3/projects/ddf27ec567494d03bd1d2469aef97264 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:45.462072 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094277e-0484-4a09-9694-e8e3937dd3d8 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:45.462363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094277e-0484-4a09-9694-e8e3937dd3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:45.462644 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094277e-0484-4a09-9694-e8e3937dd3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:45.482768 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-0094277e-0484-4a09-9694-e8e3937dd3d8 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:00:45.483974 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0094277e-0484-4a09-9694-e8e3937dd3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:45.541223 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1406/2826] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:00:45.548916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad657e2-145b-42cc-b633-77d47b785fd6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:45.549208 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad657e2-145b-42cc-b633-77d47b785fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:45.549509 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad657e2-145b-42cc-b633-77d47b785fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:45.557877 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:45.570186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2ad657e2-145b-42cc-b633-77d47b785fd6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:00:45.571392 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2ad657e2-145b-42cc-b633-77d47b785fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:45.579502 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1421/2827] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:00:45.583900 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094277e-0484-4a09-9694-e8e3937dd3d8 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:45.593695 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094277e-0484-4a09-9694-e8e3937dd3d8 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:45.600355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d5ae-00f2-4959-a444-b7a7c16737a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:45.600355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d5ae-00f2-4959-a444-b7a7c16737a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:45.600355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d5ae-00f2-4959-a444-b7a7c16737a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:45.600355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d5ae-00f2-4959-a444-b7a7c16737a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:45.600847 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d5ae-00f2-4959-a444-b7a7c16737a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:00:45.601150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d5ae-00f2-4959-a444-b7a7c16737a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:45.602782 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d5ae-00f2-4959-a444-b7a7c16737a9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:45.615054 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1407/2828] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 04:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:00:45.619260 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad657e2-145b-42cc-b633-77d47b785fd6 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:45.632968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad657e2-145b-42cc-b633-77d47b785fd6 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:45.633681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd100075-8549-4c09-9e04-93410426c05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:45.634288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd100075-8549-4c09-9e04-93410426c05f admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:45.634566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd100075-8549-4c09-9e04-93410426c05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:45.634839 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd100075-8549-4c09-9e04-93410426c05f admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:45.635464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd100075-8549-4c09-9e04-93410426c05f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:45.637370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd100075-8549-4c09-9e04-93410426c05f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:45.737436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1422/2829] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 04:00:45 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:00:45.742755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d5ae-00f2-4959-a444-b7a7c16737a9 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:45.752824 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d5ae-00f2-4959-a444-b7a7c16737a9 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:45.813001 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffde91-2004-48fe-811f-b07323ea9025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:45.814002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffde91-2004-48fe-811f-b07323ea9025 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:45.814467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffde91-2004-48fe-811f-b07323ea9025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:45.814927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffde91-2004-48fe-811f-b07323ea9025 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:45.815992 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffde91-2004-48fe-811f-b07323ea9025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:45.818927 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffde91-2004-48fe-811f-b07323ea9025 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:45.826970 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:45.913729 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1408/2830] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 04:00:45 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:00:45.918025 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd100075-8549-4c09-9e04-93410426c05f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:45.929054 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd100075-8549-4c09-9e04-93410426c05f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:45.933665 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ac762-2a3e-43fd-ac62-f972e10a055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:45.934407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ac762-2a3e-43fd-ac62-f972e10a055e admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:45.934684 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ac762-2a3e-43fd-ac62-f972e10a055e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:45.934957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ac762-2a3e-43fd-ac62-f972e10a055e admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:45.935631 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ac762-2a3e-43fd-ac62-f972e10a055e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:45.937206 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ac762-2a3e-43fd-ac62-f972e10a055e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:45.947882 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1423/2831] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:00:45.951891 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffde91-2004-48fe-811f-b07323ea9025 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:45.962418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffde91-2004-48fe-811f-b07323ea9025 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:45.967052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7818636-848f-4a21-9075-b23f62a81324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:45.967641 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7818636-848f-4a21-9075-b23f62a81324 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:45.967921 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7818636-848f-4a21-9075-b23f62a81324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:45.968196 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7818636-848f-4a21-9075-b23f62a81324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:45.968832 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7818636-848f-4a21-9075-b23f62a81324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:45.970459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7818636-848f-4a21-9075-b23f62a81324 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:45.981112 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1409/2832] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:00:45.985163 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ac762-2a3e-43fd-ac62-f972e10a055e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:45.995786 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ac762-2a3e-43fd-ac62-f972e10a055e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:46.000531 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0ca5b-6c29-4ab2-a805-9763558aec2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:46.001169 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0ca5b-6c29-4ab2-a805-9763558aec2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:46.001448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0ca5b-6c29-4ab2-a805-9763558aec2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:46.001751 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0ca5b-6c29-4ab2-a805-9763558aec2f admin admin] PATH_INFO: `/v3/projects/2c9bcc1eba7c406e95f3ab1be9d6340d/users/7d6a1c5adedb42e48f0066af4cb25090/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:46.017710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0ca5b-6c29-4ab2-a805-9763558aec2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c9bcc1eba7c406e95f3ab1be9d6340d, user_id=7d6a1c5adedb42e48f0066af4cb25090, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:46.020729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0ca5b-6c29-4ab2-a805-9763558aec2f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:46.090661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1424/2833] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:00:45 2022] PUT /identity/v3/projects/2c9bcc1eba7c406e95f3ab1be9d6340d/users/7d6a1c5adedb42e48f0066af4cb25090/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:46.101830 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80f250-c37f-4f3d-8d1c-63e3044f5a18 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:46.102113 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80f250-c37f-4f3d-8d1c-63e3044f5a18 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:46.102381 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80f250-c37f-4f3d-8d1c-63e3044f5a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:46.130625 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-cb80f250-c37f-4f3d-8d1c-63e3044f5a18 None None] MFA Rules not processed for user `7d6a1c5adedb42e48f0066af4cb25090`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:00:46.131819 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-cb80f250-c37f-4f3d-8d1c-63e3044f5a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:46.194554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1410/2834] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:00:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:00:46.198961 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0ca5b-6c29-4ab2-a805-9763558aec2f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:46.263543 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0ca5b-6c29-4ab2-a805-9763558aec2f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:46.268327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce684d8-d390-456d-b51f-dd3197adbd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:46.268996 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce684d8-d390-456d-b51f-dd3197adbd67 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:46.269278 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce684d8-d390-456d-b51f-dd3197adbd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:46.269550 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce684d8-d390-456d-b51f-dd3197adbd67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:46.270263 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce684d8-d390-456d-b51f-dd3197adbd67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:46.272159 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce684d8-d390-456d-b51f-dd3197adbd67 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:46.355350 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1425/2835] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 04:00:46 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:00:46.359629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80f250-c37f-4f3d-8d1c-63e3044f5a18 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:46.371267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80f250-c37f-4f3d-8d1c-63e3044f5a18 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:46.428435 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584e81b-4b80-4996-b122-a1c904d6163c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:46.429028 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584e81b-4b80-4996-b122-a1c904d6163c admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:46.429306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584e81b-4b80-4996-b122-a1c904d6163c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:46.429577 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584e81b-4b80-4996-b122-a1c904d6163c admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:46.430221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584e81b-4b80-4996-b122-a1c904d6163c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:46.432163 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584e81b-4b80-4996-b122-a1c904d6163c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:46.536510 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1411/2836] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 04:00:46 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:00:46.540485 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce684d8-d390-456d-b51f-dd3197adbd67 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:46.550807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce684d8-d390-456d-b51f-dd3197adbd67 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:46.556214 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffb16a-653a-4eab-8ffa-0e319d9aca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:46.556840 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffb16a-653a-4eab-8ffa-0e319d9aca1c admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:46.557122 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffb16a-653a-4eab-8ffa-0e319d9aca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:46.557394 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffb16a-653a-4eab-8ffa-0e319d9aca1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:46.558063 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffb16a-653a-4eab-8ffa-0e319d9aca1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:46.559688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffb16a-653a-4eab-8ffa-0e319d9aca1c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:46.570835 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1426/2837] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:00:46.574835 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584e81b-4b80-4996-b122-a1c904d6163c admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:46.585002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584e81b-4b80-4996-b122-a1c904d6163c admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:46.589593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a18cf-c3f7-435d-aecf-a5c551292f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:46.590293 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a18cf-c3f7-435d-aecf-a5c551292f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:46.590570 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a18cf-c3f7-435d-aecf-a5c551292f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:46.590839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a18cf-c3f7-435d-aecf-a5c551292f10 admin admin] PATH_INFO: `/v3/projects/e407ad05aeb34e64817ce996ea3bf19e/users/b9b9e4801de140b3ba2d0fb42e7ceced/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:46.605443 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a18cf-c3f7-435d-aecf-a5c551292f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e407ad05aeb34e64817ce996ea3bf19e, user_id=b9b9e4801de140b3ba2d0fb42e7ceced, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:46.608121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a18cf-c3f7-435d-aecf-a5c551292f10 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:46.685049 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1412/2838] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:00:46 2022] PUT /identity/v3/projects/e407ad05aeb34e64817ce996ea3bf19e/users/b9b9e4801de140b3ba2d0fb42e7ceced/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:00:46.689067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffb16a-653a-4eab-8ffa-0e319d9aca1c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:46.700124 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffb16a-653a-4eab-8ffa-0e319d9aca1c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:46.757779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f068b0a-58cb-48c8-ad92-f12b7b0b6b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:46.758381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f068b0a-58cb-48c8-ad92-f12b7b0b6b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:46.758664 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f068b0a-58cb-48c8-ad92-f12b7b0b6b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:46.758933 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f068b0a-58cb-48c8-ad92-f12b7b0b6b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:46.759563 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f068b0a-58cb-48c8-ad92-f12b7b0b6b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:46.761132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f068b0a-58cb-48c8-ad92-f12b7b0b6b82 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:46.770753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1427/2839] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:00:46.774958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a18cf-c3f7-435d-aecf-a5c551292f10 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:46.785048 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a18cf-c3f7-435d-aecf-a5c551292f10 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:46.794312 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e18325-bacc-4afb-ad6b-032513a35038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:46.794906 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e18325-bacc-4afb-ad6b-032513a35038 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:46.795184 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e18325-bacc-4afb-ad6b-032513a35038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:46.795457 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e18325-bacc-4afb-ad6b-032513a35038 admin admin] PATH_INFO: `/v3/projects/e407ad05aeb34e64817ce996ea3bf19e/users/b9b9e4801de140b3ba2d0fb42e7ceced/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:46.797368 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e18325-bacc-4afb-ad6b-032513a35038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e407ad05aeb34e64817ce996ea3bf19e, user_id=b9b9e4801de140b3ba2d0fb42e7ceced, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:46.800040 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e18325-bacc-4afb-ad6b-032513a35038 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:46.811475 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1413/2840] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:00:46 2022] PUT /identity/v3/projects/e407ad05aeb34e64817ce996ea3bf19e/users/b9b9e4801de140b3ba2d0fb42e7ceced/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:00:46.823200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea9fcd-30bd-4f49-81f0-5418f6d6d10a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:46.823490 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea9fcd-30bd-4f49-81f0-5418f6d6d10a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:46.823794 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea9fcd-30bd-4f49-81f0-5418f6d6d10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:46.843117 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:46.852724 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e9ea9fcd-30bd-4f49-81f0-5418f6d6d10a None None] MFA Rules not processed for user `b9b9e4801de140b3ba2d0fb42e7ceced`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:00:46.854953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e9ea9fcd-30bd-4f49-81f0-5418f6d6d10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:46.920784 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1428/2841] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:00:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:00:46.932931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e18325-bacc-4afb-ad6b-032513a35038 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:46.934636 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-d6e18325-bacc-4afb-ad6b-032513a35038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:00:46.956944 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:46.998722 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e18325-bacc-4afb-ad6b-032513a35038 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:47.003172 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc3962-fea0-4769-9e6f-53a38ac2074f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:47.003774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc3962-fea0-4769-9e6f-53a38ac2074f service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:47.004496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc3962-fea0-4769-9e6f-53a38ac2074f service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:47.005225 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc3962-fea0-4769-9e6f-53a38ac2074f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:47.014827 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc3962-fea0-4769-9e6f-53a38ac2074f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:47.016253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc3962-fea0-4769-9e6f-53a38ac2074f service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:47.026192 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1414/2842] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:00:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:00:47.226073 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea9fcd-30bd-4f49-81f0-5418f6d6d10a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:00:47.235633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea9fcd-30bd-4f49-81f0-5418f6d6d10a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:00:47.239195 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25653c7-1e99-47e7-8656-93704ef0744b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:00:47.239774 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25653c7-1e99-47e7-8656-93704ef0744b service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:00:47.240048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25653c7-1e99-47e7-8656-93704ef0744b service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:00:47.240317 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25653c7-1e99-47e7-8656-93704ef0744b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:00:47.297576 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25653c7-1e99-47e7-8656-93704ef0744b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:00:47.299064 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25653c7-1e99-47e7-8656-93704ef0744b service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:00:47.308032 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1429/2843] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:00:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:00:57.044958 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=17 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:00:57.303921 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=58 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:33.225267 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc3962-fea0-4769-9e6f-53a38ac2074f service cinder] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:33.286762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc3962-fea0-4769-9e6f-53a38ac2074f service cinder] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:33.291103 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:33.291715 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:33.291990 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:33.292258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] PATH_INFO: `/v3/users/7d6a1c5adedb42e48f0066af4cb25090` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:33.293350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6a1c5adedb42e48f0066af4cb25090)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:33.295282 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:33.458578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6a1c5adedb42e48f0066af4cb25090'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:33.501195 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6a1c5adedb42e48f0066af4cb25090'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:33.501953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6a1c5adedb42e48f0066af4cb25090'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:33.509056 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6a1c5adedb42e48f0066af4cb25090'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:33.521755 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6a1c5adedb42e48f0066af4cb25090'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:33.527611 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1415/2844] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:01:33 2022] DELETE /identity/v3/users/7d6a1c5adedb42e48f0066af4cb25090 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:33.532057 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ba689-e37d-4e41-ad1e-504da4158967 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:33.532580 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ba689-e37d-4e41-ad1e-504da4158967 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:33.532700 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ba689-e37d-4e41-ad1e-504da4158967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:33.551769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4d5ba689-e37d-4e41-ad1e-504da4158967 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:33.552883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4d5ba689-e37d-4e41-ad1e-504da4158967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:33.610188 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1430/2845] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:33.619893 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:33.621259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:33.682572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37b15d-5f9b-4f9e-88c3-2127664f5d0e admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:33.687686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051cc12-2f30-4fb1-851d-4d9347fa5fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:33.688672 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051cc12-2f30-4fb1-851d-4d9347fa5fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:33.689126 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051cc12-2f30-4fb1-851d-4d9347fa5fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:33.690133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051cc12-2f30-4fb1-851d-4d9347fa5fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:33.701346 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051cc12-2f30-4fb1-851d-4d9347fa5fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:33.703331 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051cc12-2f30-4fb1-851d-4d9347fa5fd0 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:33.714362 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1416/2846] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:01:34.120183 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ba689-e37d-4e41-ad1e-504da4158967 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:34.123458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4d5ba689-e37d-4e41-ad1e-504da4158967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:34.134695 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ba689-e37d-4e41-ad1e-504da4158967 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:34.139544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:34.140180 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:34.140459 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:34.140730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] PATH_INFO: `/v3/projects/2c9bcc1eba7c406e95f3ab1be9d6340d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:34.141890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c9bcc1eba7c406e95f3ab1be9d6340d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:34.146409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:34.244220 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c9bcc1eba7c406e95f3ab1be9d6340d', 'request_id': 'req-57c2e522-011e-4a19-8647-c34d56893f06'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:34.250314 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c9bcc1eba7c406e95f3ab1be9d6340d', 'request_id': 'req-57c2e522-011e-4a19-8647-c34d56893f06'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:34.311634 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c9bcc1eba7c406e95f3ab1be9d6340d', 'request_id': 'req-57c2e522-011e-4a19-8647-c34d56893f06'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:34.322905 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] The token cache is being invalidate because project 2c9bcc1eba7c406e95f3ab1be9d6340d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:01:34.328222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:34.328222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:34.328222 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1431/2847] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:01:34 2022] DELETE /identity/v3/projects/2c9bcc1eba7c406e95f3ab1be9d6340d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:34.329541 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051cc12-2f30-4fb1-851d-4d9347fa5fd0 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:34.339208 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051cc12-2f30-4fb1-851d-4d9347fa5fd0 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:34.356747 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:34.396150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:34.396736 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:34.397012 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:34.397291 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] PATH_INFO: `/v3/users/b9b9e4801de140b3ba2d0fb42e7ceced` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:34.398438 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9b9e4801de140b3ba2d0fb42e7ceced)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:34.400364 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:34.588955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9b9e4801de140b3ba2d0fb42e7ceced'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:34.655455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9b9e4801de140b3ba2d0fb42e7ceced'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:34.656079 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9b9e4801de140b3ba2d0fb42e7ceced'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:34.663210 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9b9e4801de140b3ba2d0fb42e7ceced'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:34.675032 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-5d020074-7ad3-4808-9e20-6002408832c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9b9e4801de140b3ba2d0fb42e7ceced'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:34.680838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1417/2848] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:01:34 2022] DELETE /identity/v3/users/b9b9e4801de140b3ba2d0fb42e7ceced => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:35.084866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:35.086342 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:35.102297 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2e522-011e-4a19-8647-c34d56893f06 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:35.163095 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:35.163719 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:35.164002 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:35.164275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] PATH_INFO: `/v3/projects/e407ad05aeb34e64817ce996ea3bf19e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:35.165393 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e407ad05aeb34e64817ce996ea3bf19e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:35.167360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:35.250191 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:35.266463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e407ad05aeb34e64817ce996ea3bf19e', 'request_id': 'req-7b389d83-a71d-4027-9ec5-b476f47d10a4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:35.273071 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e407ad05aeb34e64817ce996ea3bf19e', 'request_id': 'req-7b389d83-a71d-4027-9ec5-b476f47d10a4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:35.329589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e407ad05aeb34e64817ce996ea3bf19e', 'request_id': 'req-7b389d83-a71d-4027-9ec5-b476f47d10a4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:35.340174 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] The token cache is being invalidate because project e407ad05aeb34e64817ce996ea3bf19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:01:35.340464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:35.341091 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7b389d83-a71d-4027-9ec5-b476f47d10a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:35.342609 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1432/2849] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:01:35 2022] DELETE /identity/v3/projects/e407ad05aeb34e64817ce996ea3bf19e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:35.348171 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898a121-71f0-4ad1-912b-4de5f21a54af None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:35.348648 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898a121-71f0-4ad1-912b-4de5f21a54af None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:35.348753 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898a121-71f0-4ad1-912b-4de5f21a54af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:35.369029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-6898a121-71f0-4ad1-912b-4de5f21a54af None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:35.370296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-6898a121-71f0-4ad1-912b-4de5f21a54af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:35.430604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1418/2850] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:35.440262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a71e4-6030-48dc-98c6-2b0adab2f607 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:35.440536 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a71e4-6030-48dc-98c6-2b0adab2f607 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:35.440813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a71e4-6030-48dc-98c6-2b0adab2f607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:35.460294 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d34a71e4-6030-48dc-98c6-2b0adab2f607 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:35.461519 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d34a71e4-6030-48dc-98c6-2b0adab2f607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:35.469315 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1433/2851] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 04:01:35.474151 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898a121-71f0-4ad1-912b-4de5f21a54af None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:35.483918 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898a121-71f0-4ad1-912b-4de5f21a54af None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:35.488993 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796caa5d-1e69-45d6-a1a3-b8018aa19d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:35.489578 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796caa5d-1e69-45d6-a1a3-b8018aa19d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:35.489907 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796caa5d-1e69-45d6-a1a3-b8018aa19d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:35.490180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796caa5d-1e69-45d6-a1a3-b8018aa19d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:35.490817 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796caa5d-1e69-45d6-a1a3-b8018aa19d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:01:35.491118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796caa5d-1e69-45d6-a1a3-b8018aa19d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:35.492710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796caa5d-1e69-45d6-a1a3-b8018aa19d63 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:35.504574 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1419/2852] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 04:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:01:35.508685 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a71e4-6030-48dc-98c6-2b0adab2f607 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:35.517960 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a71e4-6030-48dc-98c6-2b0adab2f607 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:35.522917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a6b6e-9156-40d2-89fe-30528ce51ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:35.523513 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a6b6e-9156-40d2-89fe-30528ce51ecf admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:35.523797 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a6b6e-9156-40d2-89fe-30528ce51ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:35.524101 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a6b6e-9156-40d2-89fe-30528ce51ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:35.524729 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a6b6e-9156-40d2-89fe-30528ce51ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:35.526655 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a6b6e-9156-40d2-89fe-30528ce51ecf admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:35.616946 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1434/2853] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 04:01:35 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:01:35.621052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796caa5d-1e69-45d6-a1a3-b8018aa19d63 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:35.631037 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796caa5d-1e69-45d6-a1a3-b8018aa19d63 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:35.656725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:35.687434 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314703b4-4381-4d0e-b3ca-bee013a923d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:35.688041 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314703b4-4381-4d0e-b3ca-bee013a923d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:35.688319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314703b4-4381-4d0e-b3ca-bee013a923d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:35.688591 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314703b4-4381-4d0e-b3ca-bee013a923d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:35.689235 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314703b4-4381-4d0e-b3ca-bee013a923d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:35.691275 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314703b4-4381-4d0e-b3ca-bee013a923d1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:35.770378 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1420/2854] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 04:01:35 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:01:35.774438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a6b6e-9156-40d2-89fe-30528ce51ecf admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:35.786116 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a6b6e-9156-40d2-89fe-30528ce51ecf admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:35.790701 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cd6a7-ffc5-4aef-9670-23a80ad0f2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:35.791293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cd6a7-ffc5-4aef-9670-23a80ad0f2be admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:35.791596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cd6a7-ffc5-4aef-9670-23a80ad0f2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:35.791866 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cd6a7-ffc5-4aef-9670-23a80ad0f2be admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:35.792624 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cd6a7-ffc5-4aef-9670-23a80ad0f2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:35.794359 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cd6a7-ffc5-4aef-9670-23a80ad0f2be admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:35.805652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1435/2855] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:35.808934 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314703b4-4381-4d0e-b3ca-bee013a923d1 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:35.819029 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314703b4-4381-4d0e-b3ca-bee013a923d1 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:35.823538 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa327a-6d8c-4301-8dc5-ce12a44328d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:35.824150 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa327a-6d8c-4301-8dc5-ce12a44328d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:35.824422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa327a-6d8c-4301-8dc5-ce12a44328d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:35.824689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa327a-6d8c-4301-8dc5-ce12a44328d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:35.825319 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa327a-6d8c-4301-8dc5-ce12a44328d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:35.826932 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa327a-6d8c-4301-8dc5-ce12a44328d2 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:35.838099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1421/2856] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:35.842036 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cd6a7-ffc5-4aef-9670-23a80ad0f2be admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:35.853011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cd6a7-ffc5-4aef-9670-23a80ad0f2be admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:35.857668 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3964c-8b34-42fe-be9e-3da4e8b60e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:35.858337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3964c-8b34-42fe-be9e-3da4e8b60e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:35.858645 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3964c-8b34-42fe-be9e-3da4e8b60e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:35.858940 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3964c-8b34-42fe-be9e-3da4e8b60e1f admin admin] PATH_INFO: `/v3/projects/ef894c538c3547fa99e6528c194ff19a/users/3b5f6a0feb0f4a61b9ee03d4c1ac3f96/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:35.874148 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3964c-8b34-42fe-be9e-3da4e8b60e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef894c538c3547fa99e6528c194ff19a, user_id=3b5f6a0feb0f4a61b9ee03d4c1ac3f96, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:35.877406 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3964c-8b34-42fe-be9e-3da4e8b60e1f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:35.952544 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1436/2857] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:01:35 2022] PUT /identity/v3/projects/ef894c538c3547fa99e6528c194ff19a/users/3b5f6a0feb0f4a61b9ee03d4c1ac3f96/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:35.963013 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831f9f1-c653-4196-b9c7-e46baaf4d778 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:35.963314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831f9f1-c653-4196-b9c7-e46baaf4d778 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:35.963588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831f9f1-c653-4196-b9c7-e46baaf4d778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:35.992172 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a831f9f1-c653-4196-b9c7-e46baaf4d778 None None] MFA Rules not processed for user `3b5f6a0feb0f4a61b9ee03d4c1ac3f96`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:35.993416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a831f9f1-c653-4196-b9c7-e46baaf4d778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:36.056197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1422/2858] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:36.060666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3964c-8b34-42fe-be9e-3da4e8b60e1f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:36.122876 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3964c-8b34-42fe-be9e-3da4e8b60e1f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:36.127779 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab597a-82c9-4f99-904a-b2a87c3e88af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:36.128382 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab597a-82c9-4f99-904a-b2a87c3e88af admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:36.128661 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab597a-82c9-4f99-904a-b2a87c3e88af admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:36.128934 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab597a-82c9-4f99-904a-b2a87c3e88af admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:36.129553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab597a-82c9-4f99-904a-b2a87c3e88af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:36.131472 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab597a-82c9-4f99-904a-b2a87c3e88af admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:36.216597 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1437/2859] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 04:01:36 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:01:36.221321 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831f9f1-c653-4196-b9c7-e46baaf4d778 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:36.232364 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831f9f1-c653-4196-b9c7-e46baaf4d778 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:36.288589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f933d15-c575-4314-9687-35480a0c7646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:36.289180 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f933d15-c575-4314-9687-35480a0c7646 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:36.289455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f933d15-c575-4314-9687-35480a0c7646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:36.289719 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f933d15-c575-4314-9687-35480a0c7646 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:36.290363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f933d15-c575-4314-9687-35480a0c7646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:36.292296 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f933d15-c575-4314-9687-35480a0c7646 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:36.386838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1423/2860] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 04:01:36 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:01:36.390736 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab597a-82c9-4f99-904a-b2a87c3e88af admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:36.401477 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab597a-82c9-4f99-904a-b2a87c3e88af admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:36.406219 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316acf26-0343-4992-9330-aaa12af6479d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:36.409756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316acf26-0343-4992-9330-aaa12af6479d admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:36.409756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316acf26-0343-4992-9330-aaa12af6479d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:36.409756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316acf26-0343-4992-9330-aaa12af6479d admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:36.409756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316acf26-0343-4992-9330-aaa12af6479d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:36.410254 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316acf26-0343-4992-9330-aaa12af6479d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:36.420962 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1438/2861] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:36.425108 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f933d15-c575-4314-9687-35480a0c7646 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:36.435205 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f933d15-c575-4314-9687-35480a0c7646 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:36.441971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79b341-adc9-4009-a3d6-f71399840a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:36.443080 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79b341-adc9-4009-a3d6-f71399840a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:36.443371 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79b341-adc9-4009-a3d6-f71399840a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:36.443652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79b341-adc9-4009-a3d6-f71399840a37 admin admin] PATH_INFO: `/v3/projects/d9ac41beb6c645a19db751f7f2abe2c5/users/6a83078f1be34a799454eb404130ec14/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:36.460915 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79b341-adc9-4009-a3d6-f71399840a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ac41beb6c645a19db751f7f2abe2c5, user_id=6a83078f1be34a799454eb404130ec14, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:36.463607 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79b341-adc9-4009-a3d6-f71399840a37 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:36.519398 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1424/2862] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:01:36 2022] PUT /identity/v3/projects/d9ac41beb6c645a19db751f7f2abe2c5/users/6a83078f1be34a799454eb404130ec14/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:36.523467 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316acf26-0343-4992-9330-aaa12af6479d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:36.534246 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316acf26-0343-4992-9330-aaa12af6479d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:36.553883 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:36.590817 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4781c-1c13-4163-81c6-94efa3da5bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:36.591510 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4781c-1c13-4163-81c6-94efa3da5bec admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:36.591791 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4781c-1c13-4163-81c6-94efa3da5bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:36.592069 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4781c-1c13-4163-81c6-94efa3da5bec admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:36.592699 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4781c-1c13-4163-81c6-94efa3da5bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:36.594421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4781c-1c13-4163-81c6-94efa3da5bec admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:36.606004 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1439/2863] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:01:36.609908 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79b341-adc9-4009-a3d6-f71399840a37 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:36.615969 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:36.621194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79b341-adc9-4009-a3d6-f71399840a37 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:36.625860 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b656594-15db-4ffc-8540-b3da08b36ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:36.627554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b656594-15db-4ffc-8540-b3da08b36ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:36.627554 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b656594-15db-4ffc-8540-b3da08b36ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:36.627810 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b656594-15db-4ffc-8540-b3da08b36ff3 admin admin] PATH_INFO: `/v3/projects/d9ac41beb6c645a19db751f7f2abe2c5/users/6a83078f1be34a799454eb404130ec14/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:36.629780 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b656594-15db-4ffc-8540-b3da08b36ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ac41beb6c645a19db751f7f2abe2c5, user_id=6a83078f1be34a799454eb404130ec14, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:36.632447 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b656594-15db-4ffc-8540-b3da08b36ff3 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:36.643810 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1425/2864] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:01:36 2022] PUT /identity/v3/projects/d9ac41beb6c645a19db751f7f2abe2c5/users/6a83078f1be34a799454eb404130ec14/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:36.658370 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0eb00-e07a-4bfe-ae08-f9d3a0fdf13e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:36.658730 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0eb00-e07a-4bfe-ae08-f9d3a0fdf13e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:36.659007 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0eb00-e07a-4bfe-ae08-f9d3a0fdf13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:36.686526 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-97b0eb00-e07a-4bfe-ae08-f9d3a0fdf13e None None] MFA Rules not processed for user `6a83078f1be34a799454eb404130ec14`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:36.687710 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-97b0eb00-e07a-4bfe-ae08-f9d3a0fdf13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:36.754820 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1440/2865] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:36.765818 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b656594-15db-4ffc-8540-b3da08b36ff3 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:36.767537 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-9b656594-15db-4ffc-8540-b3da08b36ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:36.832422 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b656594-15db-4ffc-8540-b3da08b36ff3 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:36.839453 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19648e9-7795-4749-b9f1-f47498c1ac51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:36.840053 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19648e9-7795-4749-b9f1-f47498c1ac51 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:36.840328 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19648e9-7795-4749-b9f1-f47498c1ac51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:36.840593 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19648e9-7795-4749-b9f1-f47498c1ac51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:36.850622 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19648e9-7795-4749-b9f1-f47498c1ac51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:36.852036 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19648e9-7795-4749-b9f1-f47498c1ac51 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:36.861954 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1426/2866] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:01:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:01:36.935341 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0eb00-e07a-4bfe-ae08-f9d3a0fdf13e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:36.997598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0eb00-e07a-4bfe-ae08-f9d3a0fdf13e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:37.002965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:37.003593 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:37.003879 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:37.004157 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] PATH_INFO: `/v3/users/3b5f6a0feb0f4a61b9ee03d4c1ac3f96` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:37.005322 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b5f6a0feb0f4a61b9ee03d4c1ac3f96)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:37.007321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:37.144433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b5f6a0feb0f4a61b9ee03d4c1ac3f96'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:37.145690 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:37.159842 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b5f6a0feb0f4a61b9ee03d4c1ac3f96'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:37.231957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b5f6a0feb0f4a61b9ee03d4c1ac3f96'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:37.232589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b5f6a0feb0f4a61b9ee03d4c1ac3f96'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:37.240867 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b5f6a0feb0f4a61b9ee03d4c1ac3f96'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:37.246890 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1441/2867] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:01:36 2022] DELETE /identity/v3/users/3b5f6a0feb0f4a61b9ee03d4c1ac3f96 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:37.251327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62902da4-3583-4c2e-8163-4a69c0025a12 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:37.252268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62902da4-3583-4c2e-8163-4a69c0025a12 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:37.252553 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62902da4-3583-4c2e-8163-4a69c0025a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:37.271322 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-62902da4-3583-4c2e-8163-4a69c0025a12 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:37.272511 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-62902da4-3583-4c2e-8163-4a69c0025a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:37.330363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1427/2868] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:37.340134 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:37.341552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:37.399918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f8db2-bb45-4e0a-9b5b-27b352bf5dd8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:37.403533 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704dde9-939d-4def-b166-d2eda275db13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:37.404125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dde9-939d-4def-b166-d2eda275db13 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:37.404401 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dde9-939d-4def-b166-d2eda275db13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:37.404672 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dde9-939d-4def-b166-d2eda275db13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:37.413465 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704dde9-939d-4def-b166-d2eda275db13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:37.414860 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704dde9-939d-4def-b166-d2eda275db13 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:37.425229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1442/2869] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:01:37.850999 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62902da4-3583-4c2e-8163-4a69c0025a12 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:37.852588 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-62902da4-3583-4c2e-8163-4a69c0025a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:37.869298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62902da4-3583-4c2e-8163-4a69c0025a12 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:37.875111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:37.875762 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:37.876139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:37.876444 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] PATH_INFO: `/v3/projects/ef894c538c3547fa99e6528c194ff19a` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:37.879274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef894c538c3547fa99e6528c194ff19a)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:37.881273 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:37.999272 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef894c538c3547fa99e6528c194ff19a', 'request_id': 'req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:38.072580 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef894c538c3547fa99e6528c194ff19a', 'request_id': 'req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:38.078707 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef894c538c3547fa99e6528c194ff19a', 'request_id': 'req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:38.089920 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] The token cache is being invalidate because project ef894c538c3547fa99e6528c194ff19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:01:38.090212 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:38.090862 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:38.092360 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1428/2870] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:01:37 2022] DELETE /identity/v3/projects/ef894c538c3547fa99e6528c194ff19a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:38.096567 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704dde9-939d-4def-b166-d2eda275db13 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:38.106420 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704dde9-939d-4def-b166-d2eda275db13 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:38.163942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:38.164686 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:38.164824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:38.165092 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] PATH_INFO: `/v3/users/6a83078f1be34a799454eb404130ec14` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:38.166235 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a83078f1be34a799454eb404130ec14)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:38.168132 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:38.309452 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a83078f1be34a799454eb404130ec14'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:38.321264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a83078f1be34a799454eb404130ec14'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:38.324821 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:38.397337 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a83078f1be34a799454eb404130ec14'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:38.397999 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a83078f1be34a799454eb404130ec14'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:38.405837 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a246ea92-f078-491e-82a8-22cc18a3cab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a83078f1be34a799454eb404130ec14'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:38.412646 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1443/2871] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:01:38 2022] DELETE /identity/v3/users/6a83078f1be34a799454eb404130ec14 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:01:38.868293 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:38.869764 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:38.887689 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31a55a-66e4-44c8-9cdb-d5534e2f0b4b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:38.905364 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:38.952783 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:38.953404 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:38.953682 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:38.953988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] PATH_INFO: `/v3/projects/d9ac41beb6c645a19db751f7f2abe2c5` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:38.955100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ac41beb6c645a19db751f7f2abe2c5)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:38.958097 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:39.055641 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ac41beb6c645a19db751f7f2abe2c5', 'request_id': 'req-db6af120-d778-44fe-bae8-c926cd5b692c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:39.112612 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ac41beb6c645a19db751f7f2abe2c5', 'request_id': 'req-db6af120-d778-44fe-bae8-c926cd5b692c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:39.118355 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ac41beb6c645a19db751f7f2abe2c5', 'request_id': 'req-db6af120-d778-44fe-bae8-c926cd5b692c'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:39.130007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] The token cache is being invalidate because project d9ac41beb6c645a19db751f7f2abe2c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:01:39.130297 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:39.130950 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-db6af120-d778-44fe-bae8-c926cd5b692c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:39.132487 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1429/2872] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:01:38 2022] DELETE /identity/v3/projects/d9ac41beb6c645a19db751f7f2abe2c5 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:39.138674 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a235c03-a5d1-48d7-8e1a-6b2ba310191c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:39.139037 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a235c03-a5d1-48d7-8e1a-6b2ba310191c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:39.139386 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a235c03-a5d1-48d7-8e1a-6b2ba310191c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:39.161172 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2a235c03-a5d1-48d7-8e1a-6b2ba310191c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:39.162462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2a235c03-a5d1-48d7-8e1a-6b2ba310191c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:39.221125 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1444/2873] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:39.283120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7e939-2e5f-4fc0-8871-38e67b4f1487 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:39.283384 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7e939-2e5f-4fc0-8871-38e67b4f1487 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:39.283711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7e939-2e5f-4fc0-8871-38e67b4f1487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:39.303604 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-a7e7e939-2e5f-4fc0-8871-38e67b4f1487 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:39.304779 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a7e7e939-2e5f-4fc0-8871-38e67b4f1487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:39.312446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1430/2874] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:39.317328 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a235c03-a5d1-48d7-8e1a-6b2ba310191c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:39.326935 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a235c03-a5d1-48d7-8e1a-6b2ba310191c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:39.331421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b2f41-759c-40be-9b18-cb71d28e0df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:39.332031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b2f41-759c-40be-9b18-cb71d28e0df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:39.332327 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b2f41-759c-40be-9b18-cb71d28e0df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:39.332581 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b2f41-759c-40be-9b18-cb71d28e0df0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:39.333216 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b2f41-759c-40be-9b18-cb71d28e0df0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:01:39.333517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b2f41-759c-40be-9b18-cb71d28e0df0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:39.335127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b2f41-759c-40be-9b18-cb71d28e0df0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:39.346428 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1445/2875] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 04:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:01:39.350910 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7e939-2e5f-4fc0-8871-38e67b4f1487 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:39.359968 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7e939-2e5f-4fc0-8871-38e67b4f1487 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:39.365132 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830226b-8d19-4b56-bc2c-ae2ad3489f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:39.365735 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830226b-8d19-4b56-bc2c-ae2ad3489f78 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:39.366031 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830226b-8d19-4b56-bc2c-ae2ad3489f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:39.366304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830226b-8d19-4b56-bc2c-ae2ad3489f78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:39.366931 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830226b-8d19-4b56-bc2c-ae2ad3489f78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:39.368811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830226b-8d19-4b56-bc2c-ae2ad3489f78 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:39.433983 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1431/2876] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 04:01:39 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:01:39.438041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b2f41-759c-40be-9b18-cb71d28e0df0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:39.448692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b2f41-759c-40be-9b18-cb71d28e0df0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:39.504311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e89a7-7bc9-4a58-b8f6-8263cbe3ace4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:39.504917 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e89a7-7bc9-4a58-b8f6-8263cbe3ace4 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:39.505219 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e89a7-7bc9-4a58-b8f6-8263cbe3ace4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:39.505498 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e89a7-7bc9-4a58-b8f6-8263cbe3ace4 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:39.506162 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e89a7-7bc9-4a58-b8f6-8263cbe3ace4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:39.508058 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e89a7-7bc9-4a58-b8f6-8263cbe3ace4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:39.586807 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1446/2877] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 04:01:39 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:01:39.590973 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830226b-8d19-4b56-bc2c-ae2ad3489f78 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:39.601298 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830226b-8d19-4b56-bc2c-ae2ad3489f78 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:39.605836 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed04e7d-b465-4ef8-aad0-2e9de294ed78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:39.606503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed04e7d-b465-4ef8-aad0-2e9de294ed78 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:39.606696 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed04e7d-b465-4ef8-aad0-2e9de294ed78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:39.606959 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed04e7d-b465-4ef8-aad0-2e9de294ed78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:39.607592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed04e7d-b465-4ef8-aad0-2e9de294ed78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:39.609165 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed04e7d-b465-4ef8-aad0-2e9de294ed78 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:39.619453 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1432/2878] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:01:39.623400 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e89a7-7bc9-4a58-b8f6-8263cbe3ace4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:39.634287 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e89a7-7bc9-4a58-b8f6-8263cbe3ace4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:39.638875 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8e576-6ead-469b-af31-ca0f52914f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:39.639460 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8e576-6ead-469b-af31-ca0f52914f4c admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:39.639738 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8e576-6ead-469b-af31-ca0f52914f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:39.640013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8e576-6ead-469b-af31-ca0f52914f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:39.640652 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8e576-6ead-469b-af31-ca0f52914f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:39.642269 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8e576-6ead-469b-af31-ca0f52914f4c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:39.653333 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1447/2879] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 04:01:39.657419 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed04e7d-b465-4ef8-aad0-2e9de294ed78 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:39.668416 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed04e7d-b465-4ef8-aad0-2e9de294ed78 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:39.673099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c7f32-0e18-4a7a-8be9-03587400c9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:39.673756 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c7f32-0e18-4a7a-8be9-03587400c9cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:39.674034 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c7f32-0e18-4a7a-8be9-03587400c9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:39.674304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c7f32-0e18-4a7a-8be9-03587400c9cb admin admin] PATH_INFO: `/v3/projects/957b85963d714006ad28f738dd7b9bdc/users/202ef0b679514fe6a2aedacd1b2960cd/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:39.688624 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c7f32-0e18-4a7a-8be9-03587400c9cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=957b85963d714006ad28f738dd7b9bdc, user_id=202ef0b679514fe6a2aedacd1b2960cd, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:39.691919 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c7f32-0e18-4a7a-8be9-03587400c9cb admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:39.798601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1433/2880] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:01:39 2022] PUT /identity/v3/projects/957b85963d714006ad28f738dd7b9bdc/users/202ef0b679514fe6a2aedacd1b2960cd/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:39.808775 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125a0ee-f2d3-491a-aba1-51c57cc04443 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:39.809093 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125a0ee-f2d3-491a-aba1-51c57cc04443 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:39.809402 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125a0ee-f2d3-491a-aba1-51c57cc04443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:39.839034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f125a0ee-f2d3-491a-aba1-51c57cc04443 None None] MFA Rules not processed for user `202ef0b679514fe6a2aedacd1b2960cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:39.840280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f125a0ee-f2d3-491a-aba1-51c57cc04443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:39.881171 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:39.903651 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1448/2881] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:39.908104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c7f32-0e18-4a7a-8be9-03587400c9cb admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:39.953059 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:39.969639 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c7f32-0e18-4a7a-8be9-03587400c9cb admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:39.974347 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937ebe5-c041-46e8-b154-4e6f6425a336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:39.975455 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937ebe5-c041-46e8-b154-4e6f6425a336 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:39.975914 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937ebe5-c041-46e8-b154-4e6f6425a336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:39.976229 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937ebe5-c041-46e8-b154-4e6f6425a336 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:39.977269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937ebe5-c041-46e8-b154-4e6f6425a336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:39.979217 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937ebe5-c041-46e8-b154-4e6f6425a336 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:40.102423 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1434/2882] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 04:01:39 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:01:40.106448 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125a0ee-f2d3-491a-aba1-51c57cc04443 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:40.118083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125a0ee-f2d3-491a-aba1-51c57cc04443 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:40.175241 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754a71d-e471-410b-9a19-ed35bd62b57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:40.176360 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754a71d-e471-410b-9a19-ed35bd62b57b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:40.176790 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754a71d-e471-410b-9a19-ed35bd62b57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:40.177198 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754a71d-e471-410b-9a19-ed35bd62b57b admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:40.177856 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754a71d-e471-410b-9a19-ed35bd62b57b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:40.179781 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754a71d-e471-410b-9a19-ed35bd62b57b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:40.303295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1449/2883] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 04:01:40 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:01:40.307237 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937ebe5-c041-46e8-b154-4e6f6425a336 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:40.318264 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937ebe5-c041-46e8-b154-4e6f6425a336 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:40.322821 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefb09b-3897-4d05-a8da-37efeed5cd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:40.323401 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefb09b-3897-4d05-a8da-37efeed5cd8b admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:40.324099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefb09b-3897-4d05-a8da-37efeed5cd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:40.324099 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefb09b-3897-4d05-a8da-37efeed5cd8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:40.324573 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefb09b-3897-4d05-a8da-37efeed5cd8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:40.326197 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefb09b-3897-4d05-a8da-37efeed5cd8b admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:40.336426 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1435/2884] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:40.340725 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754a71d-e471-410b-9a19-ed35bd62b57b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:40.351330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754a71d-e471-410b-9a19-ed35bd62b57b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:40.355986 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5d907-ca1a-46ca-991a-da894d3b72bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:40.356639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5d907-ca1a-46ca-991a-da894d3b72bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:40.356923 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5d907-ca1a-46ca-991a-da894d3b72bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:40.357200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5d907-ca1a-46ca-991a-da894d3b72bb admin admin] PATH_INFO: `/v3/projects/ad6e7a9e4bc841f48e2d13d3af1ba8f9/users/e327173d638e48d19bfd83de83bf270f/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:40.372555 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5d907-ca1a-46ca-991a-da894d3b72bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6e7a9e4bc841f48e2d13d3af1ba8f9, user_id=e327173d638e48d19bfd83de83bf270f, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:40.375261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5d907-ca1a-46ca-991a-da894d3b72bb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:40.449445 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1450/2885] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:01:40 2022] PUT /identity/v3/projects/ad6e7a9e4bc841f48e2d13d3af1ba8f9/users/e327173d638e48d19bfd83de83bf270f/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:40.453446 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefb09b-3897-4d05-a8da-37efeed5cd8b admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:40.469421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefb09b-3897-4d05-a8da-37efeed5cd8b admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:40.528274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cb343-5243-43bc-aa07-97ec14635b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:40.528879 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cb343-5243-43bc-aa07-97ec14635b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:40.529160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cb343-5243-43bc-aa07-97ec14635b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:40.529430 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cb343-5243-43bc-aa07-97ec14635b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:40.530111 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cb343-5243-43bc-aa07-97ec14635b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:40.531710 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cb343-5243-43bc-aa07-97ec14635b1e admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:40.542009 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1436/2886] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:40.546288 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5d907-ca1a-46ca-991a-da894d3b72bb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:40.557405 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5d907-ca1a-46ca-991a-da894d3b72bb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:40.561823 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47caac3-929e-41bd-901b-c1883fa36bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:40.562463 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47caac3-929e-41bd-901b-c1883fa36bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:40.562758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47caac3-929e-41bd-901b-c1883fa36bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:40.563029 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47caac3-929e-41bd-901b-c1883fa36bcd admin admin] PATH_INFO: `/v3/projects/ad6e7a9e4bc841f48e2d13d3af1ba8f9/users/e327173d638e48d19bfd83de83bf270f/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:40.564902 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47caac3-929e-41bd-901b-c1883fa36bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6e7a9e4bc841f48e2d13d3af1ba8f9, user_id=e327173d638e48d19bfd83de83bf270f, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:40.567589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47caac3-929e-41bd-901b-c1883fa36bcd admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:40.577501 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1451/2887] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:01:40 2022] PUT /identity/v3/projects/ad6e7a9e4bc841f48e2d13d3af1ba8f9/users/e327173d638e48d19bfd83de83bf270f/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:40.587206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a13d-2479-4402-9093-bd13e730a8d4 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:40.587608 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a13d-2479-4402-9093-bd13e730a8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:40.587890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a13d-2479-4402-9093-bd13e730a8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:40.616638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-7e77a13d-2479-4402-9093-bd13e730a8d4 None None] MFA Rules not processed for user `e327173d638e48d19bfd83de83bf270f`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:40.617861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-7e77a13d-2479-4402-9093-bd13e730a8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:40.684209 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1437/2888] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:40.694884 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47caac3-929e-41bd-901b-c1883fa36bcd admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:40.696589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f47caac3-929e-41bd-901b-c1883fa36bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:40.756809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47caac3-929e-41bd-901b-c1883fa36bcd admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:40.760432 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280e566-999c-44a8-abf8-5b91661170a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:40.761023 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280e566-999c-44a8-abf8-5b91661170a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:40.761300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280e566-999c-44a8-abf8-5b91661170a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:40.761571 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280e566-999c-44a8-abf8-5b91661170a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:40.770841 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280e566-999c-44a8-abf8-5b91661170a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:40.772280 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280e566-999c-44a8-abf8-5b91661170a2 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:40.781433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1452/2889] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:01:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:01:41.321327 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77a13d-2479-4402-9093-bd13e730a8d4 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:41.360839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:41.381677 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77a13d-2479-4402-9093-bd13e730a8d4 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:41.386160 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:41.386712 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:41.386985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:41.387253 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] PATH_INFO: `/v3/users/202ef0b679514fe6a2aedacd1b2960cd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:41.388349 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=202ef0b679514fe6a2aedacd1b2960cd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:41.390293 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:41.545396 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '202ef0b679514fe6a2aedacd1b2960cd'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:41.615193 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '202ef0b679514fe6a2aedacd1b2960cd'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:41.615841 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '202ef0b679514fe6a2aedacd1b2960cd'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:41.623060 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '202ef0b679514fe6a2aedacd1b2960cd'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:41.635775 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '202ef0b679514fe6a2aedacd1b2960cd'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:41.641662 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1438/2890] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:01:41 2022] DELETE /identity/v3/users/202ef0b679514fe6a2aedacd1b2960cd => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:41.646552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafc1f5-bf77-4a6c-bc1d-caf31d16d688 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:41.647048 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafc1f5-bf77-4a6c-bc1d-caf31d16d688 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:41.647154 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafc1f5-bf77-4a6c-bc1d-caf31d16d688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:41.666862 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9fafc1f5-bf77-4a6c-bc1d-caf31d16d688 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:41.668066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9fafc1f5-bf77-4a6c-bc1d-caf31d16d688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:41.700891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:41.728614 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1453/2891] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:41.738833 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:41.740276 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:41.797811 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0335c21-8ab1-4d51-a27d-a728db17ce2d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:41.802725 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d3fae-8daf-41df-9b19-c0d022d784b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:41.803324 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d3fae-8daf-41df-9b19-c0d022d784b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:41.803601 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d3fae-8daf-41df-9b19-c0d022d784b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:41.803869 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d3fae-8daf-41df-9b19-c0d022d784b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:41.813128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d3fae-8daf-41df-9b19-c0d022d784b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:41.814575 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d3fae-8daf-41df-9b19-c0d022d784b5 service neutron] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:41.828006 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1439/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:01:42.239654 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafc1f5-bf77-4a6c-bc1d-caf31d16d688 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:42.241066 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9fafc1f5-bf77-4a6c-bc1d-caf31d16d688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:42.256965 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafc1f5-bf77-4a6c-bc1d-caf31d16d688 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:42.262451 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:42.263439 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:42.263778 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:42.264123 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] PATH_INFO: `/v3/projects/957b85963d714006ad28f738dd7b9bdc` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:42.265229 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] RBAC: Authorizing `identity:delete_project(project_id=957b85963d714006ad28f738dd7b9bdc)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:42.267107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:42.404276 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '957b85963d714006ad28f738dd7b9bdc', 'request_id': 'req-64dc06f6-067d-4e36-b37a-6f17ee42046f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:42.410573 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '957b85963d714006ad28f738dd7b9bdc', 'request_id': 'req-64dc06f6-067d-4e36-b37a-6f17ee42046f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:42.478446 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '957b85963d714006ad28f738dd7b9bdc', 'request_id': 'req-64dc06f6-067d-4e36-b37a-6f17ee42046f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:42.491789 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] The token cache is being invalidate because project 957b85963d714006ad28f738dd7b9bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:01:42.492107 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:42.492758 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:42.494425 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1454/2893] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:01:42 2022] DELETE /identity/v3/projects/957b85963d714006ad28f738dd7b9bdc => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:42.498617 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d3fae-8daf-41df-9b19-c0d022d784b5 service neutron] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:42.509496 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d3fae-8daf-41df-9b19-c0d022d784b5 service neutron] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:42.564838 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:42.565459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:42.565763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:42.566095 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] PATH_INFO: `/v3/users/e327173d638e48d19bfd83de83bf270f` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:42.567955 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e327173d638e48d19bfd83de83bf270f)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:42.569894 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:42.571332 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:42.713783 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e327173d638e48d19bfd83de83bf270f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:42.787259 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e327173d638e48d19bfd83de83bf270f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:42.787948 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e327173d638e48d19bfd83de83bf270f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:42.797412 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e327173d638e48d19bfd83de83bf270f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:42.816261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-02545c5d-a6ed-4bff-b6f6-4fee4dc98ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e327173d638e48d19bfd83de83bf270f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:42.822294 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1440/2894] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:01:42 2022] DELETE /identity/v3/users/e327173d638e48d19bfd83de83bf270f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:43.200577 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:43.202194 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:43.215861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc06f6-067d-4e36-b37a-6f17ee42046f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:43.276290 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:43.276974 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:43.277261 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:43.277538 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] PATH_INFO: `/v3/projects/ad6e7a9e4bc841f48e2d13d3af1ba8f9` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:43.278688 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6e7a9e4bc841f48e2d13d3af1ba8f9)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:43.280666 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:43.370301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6e7a9e4bc841f48e2d13d3af1ba8f9', 'request_id': 'req-3255c3f3-e452-4091-b713-8133b86009df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:43.376408 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6e7a9e4bc841f48e2d13d3af1ba8f9', 'request_id': 'req-3255c3f3-e452-4091-b713-8133b86009df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:43.437740 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6e7a9e4bc841f48e2d13d3af1ba8f9', 'request_id': 'req-3255c3f3-e452-4091-b713-8133b86009df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:43.439805 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:43.448678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] The token cache is being invalidate because project ad6e7a9e4bc841f48e2d13d3af1ba8f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:01:43.448968 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:43.449595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3255c3f3-e452-4091-b713-8133b86009df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:43.451111 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1455/2895] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:01:43 2022] DELETE /identity/v3/projects/ad6e7a9e4bc841f48e2d13d3af1ba8f9 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:43.456124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c05a5-754e-4b76-af61-f97445f7e412 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:43.460909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c05a5-754e-4b76-af61-f97445f7e412 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:43.460909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c05a5-754e-4b76-af61-f97445f7e412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:43.478002 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-fb8c05a5-754e-4b76-af61-f97445f7e412 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:43.479194 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-fb8c05a5-754e-4b76-af61-f97445f7e412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:43.538314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1441/2896] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:43.546083 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c3c4-0e72-4421-8d14-2fb22d078da0 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:43.546366 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c3c4-0e72-4421-8d14-2fb22d078da0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:43.546640 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c3c4-0e72-4421-8d14-2fb22d078da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:43.566756 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-88e4c3c4-0e72-4421-8d14-2fb22d078da0 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:43.567942 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-88e4c3c4-0e72-4421-8d14-2fb22d078da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:43.575809 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1456/2897] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:43.580104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c05a5-754e-4b76-af61-f97445f7e412 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:43.589687 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c05a5-754e-4b76-af61-f97445f7e412 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:43.594152 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266244a-4e7c-4bd7-a821-617a1d8fd0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:43.594734 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266244a-4e7c-4bd7-a821-617a1d8fd0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:43.595007 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266244a-4e7c-4bd7-a821-617a1d8fd0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:43.595274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266244a-4e7c-4bd7-a821-617a1d8fd0a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:43.595908 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266244a-4e7c-4bd7-a821-617a1d8fd0a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81726) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:01:43.596202 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266244a-4e7c-4bd7-a821-617a1d8fd0a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:43.598418 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266244a-4e7c-4bd7-a821-617a1d8fd0a5 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:43.610236 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1442/2898] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 04:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 04:01:43.615217 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c3c4-0e72-4421-8d14-2fb22d078da0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:43.626339 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c3c4-0e72-4421-8d14-2fb22d078da0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:43.632969 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ccb2f-8645-4cdf-8eba-f9e91d6f1e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:43.633553 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ccb2f-8645-4cdf-8eba-f9e91d6f1e91 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:43.633861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ccb2f-8645-4cdf-8eba-f9e91d6f1e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:43.634176 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ccb2f-8645-4cdf-8eba-f9e91d6f1e91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:43.634851 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ccb2f-8645-4cdf-8eba-f9e91d6f1e91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:43.636783 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ccb2f-8645-4cdf-8eba-f9e91d6f1e91 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:43.834625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1457/2899] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 04:01:43 2022] POST /identity/v3/projects => generated 375 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:01:43.838740 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266244a-4e7c-4bd7-a821-617a1d8fd0a5 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:43.849120 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266244a-4e7c-4bd7-a821-617a1d8fd0a5 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:43.903923 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc5dc9-d60c-43ea-a07d-2aa71854e820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:43.904509 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc5dc9-d60c-43ea-a07d-2aa71854e820 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:43.904784 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc5dc9-d60c-43ea-a07d-2aa71854e820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:43.905051 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc5dc9-d60c-43ea-a07d-2aa71854e820 admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:43.905670 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc5dc9-d60c-43ea-a07d-2aa71854e820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:43.907587 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc5dc9-d60c-43ea-a07d-2aa71854e820 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:43.918994 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:43.987131 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1443/2900] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 04:01:43 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:01:43.991320 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ccb2f-8645-4cdf-8eba-f9e91d6f1e91 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:44.001716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ccb2f-8645-4cdf-8eba-f9e91d6f1e91 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:44.006340 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9494-4af9-4126-89e0-df6dfab88121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:44.010561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9494-4af9-4126-89e0-df6dfab88121 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.010561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9494-4af9-4126-89e0-df6dfab88121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.010561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9494-4af9-4126-89e0-df6dfab88121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.010561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d9494-4af9-4126-89e0-df6dfab88121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:44.010561 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d9494-4af9-4126-89e0-df6dfab88121 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:44.020059 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1458/2901] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:44.024063 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc5dc9-d60c-43ea-a07d-2aa71854e820 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:44.034388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc5dc9-d60c-43ea-a07d-2aa71854e820 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:44.040030 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d171318-e3b1-49d4-ac8d-6a811b7455c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:44.040610 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d171318-e3b1-49d4-ac8d-6a811b7455c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.040882 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d171318-e3b1-49d4-ac8d-6a811b7455c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.041147 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d171318-e3b1-49d4-ac8d-6a811b7455c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.041815 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d171318-e3b1-49d4-ac8d-6a811b7455c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:44.043493 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d171318-e3b1-49d4-ac8d-6a811b7455c8 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:44.055269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1444/2902] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:44.059293 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9494-4af9-4126-89e0-df6dfab88121 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:44.069242 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9494-4af9-4126-89e0-df6dfab88121 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:44.073799 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bf074-b4c6-426b-aacd-0333e6be4b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:44.074433 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bf074-b4c6-426b-aacd-0333e6be4b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.074714 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bf074-b4c6-426b-aacd-0333e6be4b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.074986 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bf074-b4c6-426b-aacd-0333e6be4b49 admin admin] PATH_INFO: `/v3/projects/619f144451ff4f5db7c537a1116694aa/users/6e3fe62b76c44757a34b43ca749f3d6a/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.089564 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bf074-b4c6-426b-aacd-0333e6be4b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=619f144451ff4f5db7c537a1116694aa, user_id=6e3fe62b76c44757a34b43ca749f3d6a, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:44.092250 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bf074-b4c6-426b-aacd-0333e6be4b49 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:44.141436 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1459/2903] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:01:44 2022] PUT /identity/v3/projects/619f144451ff4f5db7c537a1116694aa/users/6e3fe62b76c44757a34b43ca749f3d6a/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:44.152572 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d3dec-142b-458a-8df3-4b8ff7009ded None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.158245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d3dec-142b-458a-8df3-4b8ff7009ded None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.158245 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d3dec-142b-458a-8df3-4b8ff7009ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.182166 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-ff1d3dec-142b-458a-8df3-4b8ff7009ded None None] MFA Rules not processed for user `6e3fe62b76c44757a34b43ca749f3d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:44.183350 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ff1d3dec-142b-458a-8df3-4b8ff7009ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:44.247584 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1445/2904] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:44.252109 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bf074-b4c6-426b-aacd-0333e6be4b49 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:44.313024 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bf074-b4c6-426b-aacd-0333e6be4b49 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:44.317836 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407142-5cea-4444-b97c-4d6d575af155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:44.318456 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407142-5cea-4444-b97c-4d6d575af155 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.318743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407142-5cea-4444-b97c-4d6d575af155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.319016 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407142-5cea-4444-b97c-4d6d575af155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.319639 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46407142-5cea-4444-b97c-4d6d575af155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:44.321568 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46407142-5cea-4444-b97c-4d6d575af155 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:44.423430 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1460/2905] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 04:01:44 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:01:44.427525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d3dec-142b-458a-8df3-4b8ff7009ded None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:44.438428 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d3dec-142b-458a-8df3-4b8ff7009ded None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:44.497158 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec2ce7-d9bc-4014-9bcf-caa7f72d244f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:44.497845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec2ce7-d9bc-4014-9bcf-caa7f72d244f admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.498139 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec2ce7-d9bc-4014-9bcf-caa7f72d244f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.498410 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec2ce7-d9bc-4014-9bcf-caa7f72d244f admin admin] PATH_INFO: `/v3/users` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.499050 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec2ce7-d9bc-4014-9bcf-caa7f72d244f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:44.500971 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec2ce7-d9bc-4014-9bcf-caa7f72d244f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:44.599121 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1446/2906] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 04:01:44 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:01:44.603625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407142-5cea-4444-b97c-4d6d575af155 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:44.614669 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407142-5cea-4444-b97c-4d6d575af155 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:44.619455 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c44680-1370-4b04-8c62-9be5f03226d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:44.620027 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44680-1370-4b04-8c62-9be5f03226d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.620305 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44680-1370-4b04-8c62-9be5f03226d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.620603 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44680-1370-4b04-8c62-9be5f03226d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.621262 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c44680-1370-4b04-8c62-9be5f03226d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:44.622949 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c44680-1370-4b04-8c62-9be5f03226d0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:44.633681 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1461/2907] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:44.640568 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec2ce7-d9bc-4014-9bcf-caa7f72d244f admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:44.651006 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec2ce7-d9bc-4014-9bcf-caa7f72d244f admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:44.655708 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cd06d-9b2d-4207-9634-56b2f1a85583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:44.656339 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cd06d-9b2d-4207-9634-56b2f1a85583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.656618 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cd06d-9b2d-4207-9634-56b2f1a85583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.656890 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cd06d-9b2d-4207-9634-56b2f1a85583 admin admin] PATH_INFO: `/v3/projects/1fb5d7d8fd19418896099af4df8b80c5/users/978285d29e42479f890c23eb0318dcb7/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.672104 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cd06d-9b2d-4207-9634-56b2f1a85583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb5d7d8fd19418896099af4df8b80c5, user_id=978285d29e42479f890c23eb0318dcb7, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:44.674839 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cd06d-9b2d-4207-9634-56b2f1a85583 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:44.730277 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1447/2908] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:01:44 2022] PUT /identity/v3/projects/1fb5d7d8fd19418896099af4df8b80c5/users/978285d29e42479f890c23eb0318dcb7/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:01:44.734108 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c44680-1370-4b04-8c62-9be5f03226d0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:44.744186 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c44680-1370-4b04-8c62-9be5f03226d0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:44.800416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469395d-6ef2-4ccf-b649-80cf6c5bd190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:44.801034 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469395d-6ef2-4ccf-b649-80cf6c5bd190 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.801311 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469395d-6ef2-4ccf-b649-80cf6c5bd190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.801589 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469395d-6ef2-4ccf-b649-80cf6c5bd190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.802255 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469395d-6ef2-4ccf-b649-80cf6c5bd190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:44.803916 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469395d-6ef2-4ccf-b649-80cf6c5bd190 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:44.806110 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:44.814487 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1462/2909] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:44.829369 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cd06d-9b2d-4207-9634-56b2f1a85583 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:44.840207 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cd06d-9b2d-4207-9634-56b2f1a85583 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:44.845011 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea727cc2-de14-46a1-8b38-04643d2c757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:44.845652 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea727cc2-de14-46a1-8b38-04643d2c757d admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.845964 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea727cc2-de14-46a1-8b38-04643d2c757d admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.846240 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea727cc2-de14-46a1-8b38-04643d2c757d admin admin] PATH_INFO: `/v3/projects/1fb5d7d8fd19418896099af4df8b80c5/users/978285d29e42479f890c23eb0318dcb7/roles/33ff6e45fa82455ab009b3b37a581e49` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.848925 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea727cc2-de14-46a1-8b38-04643d2c757d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb5d7d8fd19418896099af4df8b80c5, user_id=978285d29e42479f890c23eb0318dcb7, role_id=33ff6e45fa82455ab009b3b37a581e49)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:44.851841 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea727cc2-de14-46a1-8b38-04643d2c757d admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:44.862070 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1448/2910] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:01:44 2022] PUT /identity/v3/projects/1fb5d7d8fd19418896099af4df8b80c5/users/978285d29e42479f890c23eb0318dcb7/roles/33ff6e45fa82455ab009b3b37a581e49 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:44.873479 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4004c-7813-4dc6-9ba2-946b7ef38a1f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:44.873870 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4004c-7813-4dc6-9ba2-946b7ef38a1f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:44.874153 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4004c-7813-4dc6-9ba2-946b7ef38a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:44.902619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b4c4004c-7813-4dc6-9ba2-946b7ef38a1f None None] MFA Rules not processed for user `978285d29e42479f890c23eb0318dcb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:44.903838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b4c4004c-7813-4dc6-9ba2-946b7ef38a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:44.971319 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1463/2911] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:44.982851 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea727cc2-de14-46a1-8b38-04643d2c757d admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:44.984525 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-ea727cc2-de14-46a1-8b38-04643d2c757d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:45.015737 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:45.043804 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea727cc2-de14-46a1-8b38-04643d2c757d admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:45.047565 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9afbe47-bede-4e36-948e-0f948812223f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:45.050218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afbe47-bede-4e36-948e-0f948812223f service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:45.050218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afbe47-bede-4e36-948e-0f948812223f service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:45.050218 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afbe47-bede-4e36-948e-0f948812223f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:45.105064 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9afbe47-bede-4e36-948e-0f948812223f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:45.106517 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9afbe47-bede-4e36-948e-0f948812223f service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:45.115249 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1449/2912] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:01:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:01:48.080551 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4004c-7813-4dc6-9ba2-946b7ef38a1f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:48.142131 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4004c-7813-4dc6-9ba2-946b7ef38a1f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:48.147566 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:48.148181 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:48.148462 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:48.148737 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] PATH_INFO: `/v3/users/6e3fe62b76c44757a34b43ca749f3d6a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:48.149900 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3fe62b76c44757a34b43ca749f3d6a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:48.151813 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:48.285464 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3fe62b76c44757a34b43ca749f3d6a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:48.297838 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3fe62b76c44757a34b43ca749f3d6a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:48.386200 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3fe62b76c44757a34b43ca749f3d6a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:48.386764 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3fe62b76c44757a34b43ca749f3d6a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:48.394127 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3fe62b76c44757a34b43ca749f3d6a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:48.400428 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1464/2913] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:01:48 2022] DELETE /identity/v3/users/6e3fe62b76c44757a34b43ca749f3d6a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:48.404986 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3901a-0f86-4f83-8de7-0034f1e5c13b None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:48.405467 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3901a-0f86-4f83-8de7-0034f1e5c13b None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:48.405589 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3901a-0f86-4f83-8de7-0034f1e5c13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:48.424100 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-55b3901a-0f86-4f83-8de7-0034f1e5c13b None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:48.425279 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-55b3901a-0f86-4f83-8de7-0034f1e5c13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:48.483302 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1450/2914] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:48.493517 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:48.494957 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:48.500421 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:48.558128 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304443c-bf25-484a-9b76-2d9fb8074532 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:48.562019 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a1d3-09d5-4a6a-b275-59f6ebe6f1d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:48.562638 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a1d3-09d5-4a6a-b275-59f6ebe6f1d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:48.562918 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a1d3-09d5-4a6a-b275-59f6ebe6f1d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:48.563192 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a1d3-09d5-4a6a-b275-59f6ebe6f1d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:48.573031 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00a1d3-09d5-4a6a-b275-59f6ebe6f1d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:48.574489 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00a1d3-09d5-4a6a-b275-59f6ebe6f1d6 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:48.584275 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1465/2915] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:01:49.011343 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3901a-0f86-4f83-8de7-0034f1e5c13b None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:49.012876 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-55b3901a-0f86-4f83-8de7-0034f1e5c13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:49.026000 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3901a-0f86-4f83-8de7-0034f1e5c13b None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:49.031421 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:49.032128 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:49.032534 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:49.032829 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] PATH_INFO: `/v3/projects/619f144451ff4f5db7c537a1116694aa` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:49.033996 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=619f144451ff4f5db7c537a1116694aa)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:49.035938 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:49.142409 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '619f144451ff4f5db7c537a1116694aa', 'request_id': 'req-4d288ca2-ee85-4575-913b-a3ccc5506ad9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:49.202909 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '619f144451ff4f5db7c537a1116694aa', 'request_id': 'req-4d288ca2-ee85-4575-913b-a3ccc5506ad9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:49.210529 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '619f144451ff4f5db7c537a1116694aa', 'request_id': 'req-4d288ca2-ee85-4575-913b-a3ccc5506ad9'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:49.222261 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] The token cache is being invalidate because project 619f144451ff4f5db7c537a1116694aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:01:49.222548 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:49.223200 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:49.225649 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1451/2916] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:01:49 2022] DELETE /identity/v3/projects/619f144451ff4f5db7c537a1116694aa => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:49.229596 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a1d3-09d5-4a6a-b275-59f6ebe6f1d6 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:49.239953 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a1d3-09d5-4a6a-b275-59f6ebe6f1d6 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:49.295716 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:49.296330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:49.296607 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:49.296878 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] PATH_INFO: `/v3/users/978285d29e42479f890c23eb0318dcb7` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:49.298000 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=978285d29e42479f890c23eb0318dcb7)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:49.299896 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:49.462855 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '978285d29e42479f890c23eb0318dcb7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:49.475385 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '978285d29e42479f890c23eb0318dcb7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:49.551893 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '978285d29e42479f890c23eb0318dcb7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:49.552515 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '978285d29e42479f890c23eb0318dcb7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:49.560238 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fa8ea381-35a8-45f0-9bba-ad52866f89e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '978285d29e42479f890c23eb0318dcb7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:49.566207 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1466/2917] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:01:49 2022] DELETE /identity/v3/users/978285d29e42479f890c23eb0318dcb7 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:50.050481 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:50.051953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:50.065141 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:50.070345 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d288ca2-ee85-4575-913b-a3ccc5506ad9 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:50.127606 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:50.128268 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:50.128749 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:50.129274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] PATH_INFO: `/v3/projects/1fb5d7d8fd19418896099af4df8b80c5` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:50.130834 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb5d7d8fd19418896099af4df8b80c5)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:50.132988 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:50.222211 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb5d7d8fd19418896099af4df8b80c5', 'request_id': 'req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:50.273191 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb5d7d8fd19418896099af4df8b80c5', 'request_id': 'req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:50.279025 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb5d7d8fd19418896099af4df8b80c5', 'request_id': 'req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:50.290388 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] The token cache is being invalidate because project 1fb5d7d8fd19418896099af4df8b80c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:01:50.290686 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:50.291313 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-3d2e797f-65d3-4d2b-80c2-f298ed4a682f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:50.292774 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1452/2918] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:01:50 2022] DELETE /identity/v3/projects/1fb5d7d8fd19418896099af4df8b80c5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:50.297760 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85309436-d037-488b-b9ac-0b46c76f687c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:50.298248 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85309436-d037-488b-b9ac-0b46c76f687c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:50.298380 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85309436-d037-488b-b9ac-0b46c76f687c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:50.319196 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-85309436-d037-488b-b9ac-0b46c76f687c None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:50.320396 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-85309436-d037-488b-b9ac-0b46c76f687c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:50.339026 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:50.378503 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1467/2919] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:50.386790 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c905-031e-49c1-b40e-3d7fc8531bc6 None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:50.387124 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c905-031e-49c1-b40e-3d7fc8531bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:50.387351 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c905-031e-49c1-b40e-3d7fc8531bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:50.408166 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-49a5c905-031e-49c1-b40e-3d7fc8531bc6 None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:50.409370 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-49a5c905-031e-49c1-b40e-3d7fc8531bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:50.417520 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1453/2920] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:50.421824 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85309436-d037-488b-b9ac-0b46c76f687c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:50.431409 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85309436-d037-488b-b9ac-0b46c76f687c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:50.436494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03155723-43a6-479a-97fa-3839e7a72f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:50.437079 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03155723-43a6-479a-97fa-3839e7a72f30 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:50.437357 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03155723-43a6-479a-97fa-3839e7a72f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:50.437627 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03155723-43a6-479a-97fa-3839e7a72f30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:50.438295 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03155723-43a6-479a-97fa-3839e7a72f30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 04:01:50.438595 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03155723-43a6-479a-97fa-3839e7a72f30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:50.440173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03155723-43a6-479a-97fa-3839e7a72f30 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:50.452264 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1468/2921] 10.222.0.42 () {58 vars in 1153 bytes} [Sat Sep 17 04:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 04:01:50.456459 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5c905-031e-49c1-b40e-3d7fc8531bc6 None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:50.465921 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5c905-031e-49c1-b40e-3d7fc8531bc6 None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:50.470501 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983a424-478f-480d-988b-4c565f1f6f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:50.471084 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983a424-478f-480d-988b-4c565f1f6f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:50.471360 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983a424-478f-480d-988b-4c565f1f6f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:50.471628 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983a424-478f-480d-988b-4c565f1f6f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:50.472269 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983a424-478f-480d-988b-4c565f1f6f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:50.474206 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983a424-478f-480d-988b-4c565f1f6f44 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:50.554552 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1454/2922] 10.222.0.42 () {60 vars in 1153 bytes} [Sat Sep 17 04:01:50 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 04:01:50.558625 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03155723-43a6-479a-97fa-3839e7a72f30 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:50.568598 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03155723-43a6-479a-97fa-3839e7a72f30 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:50.625471 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818dc5e-fd21-47b4-a727-459bd5a4201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:50.626103 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818dc5e-fd21-47b4-a727-459bd5a4201a admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:50.626381 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818dc5e-fd21-47b4-a727-459bd5a4201a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:50.626678 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818dc5e-fd21-47b4-a727-459bd5a4201a admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:50.627300 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818dc5e-fd21-47b4-a727-459bd5a4201a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:50.629184 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818dc5e-fd21-47b4-a727-459bd5a4201a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:50.712552 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1469/2923] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Sep 17 04:01:50 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 04:01:50.716559 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983a424-478f-480d-988b-4c565f1f6f44 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:50.727953 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983a424-478f-480d-988b-4c565f1f6f44 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:50.732724 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e9939-ffa2-412f-8dda-d63425e8ab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:50.733306 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e9939-ffa2-412f-8dda-d63425e8ab49 admin admin] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:50.733581 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e9939-ffa2-412f-8dda-d63425e8ab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:50.733879 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e9939-ffa2-412f-8dda-d63425e8ab49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:50.734508 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e9939-ffa2-412f-8dda-d63425e8ab49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:50.736091 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e9939-ffa2-412f-8dda-d63425e8ab49 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:50.745861 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1455/2924] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:50.749804 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818dc5e-fd21-47b4-a727-459bd5a4201a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:50.759743 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818dc5e-fd21-47b4-a727-459bd5a4201a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:50.764486 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22531b0e-7b81-4c9d-a36a-005c0689ddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:50.765067 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22531b0e-7b81-4c9d-a36a-005c0689ddff admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:50.765354 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22531b0e-7b81-4c9d-a36a-005c0689ddff admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:50.765633 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22531b0e-7b81-4c9d-a36a-005c0689ddff admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:50.766301 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22531b0e-7b81-4c9d-a36a-005c0689ddff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:50.767891 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22531b0e-7b81-4c9d-a36a-005c0689ddff admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:50.779173 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1470/2925] 10.222.0.42 () {58 vars in 1122 bytes} [Sat Sep 17 04:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 04:01:50.783118 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e9939-ffa2-412f-8dda-d63425e8ab49 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:50.793474 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e9939-ffa2-412f-8dda-d63425e8ab49 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:50.798052 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ccfdc-4a0e-43cb-8ea7-c516ad1501c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:50.798688 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ccfdc-4a0e-43cb-8ea7-c516ad1501c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:50.798962 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ccfdc-4a0e-43cb-8ea7-c516ad1501c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:50.799234 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ccfdc-4a0e-43cb-8ea7-c516ad1501c1 admin admin] PATH_INFO: `/v3/projects/0bf7d18b6eae44589fc5ff45b4e1fd93/users/89243b6377da407288c165f001a14c15/roles/d801ebea910445a7a6100b35e07630dd` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:50.814142 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ccfdc-4a0e-43cb-8ea7-c516ad1501c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bf7d18b6eae44589fc5ff45b4e1fd93, user_id=89243b6377da407288c165f001a14c15, role_id=d801ebea910445a7a6100b35e07630dd)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:50.816711 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ccfdc-4a0e-43cb-8ea7-c516ad1501c1 admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:50.878133 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1456/2926] 10.222.0.42 () {60 vars in 1483 bytes} [Sat Sep 17 04:01:50 2022] PUT /identity/v3/projects/0bf7d18b6eae44589fc5ff45b4e1fd93/users/89243b6377da407288c165f001a14c15/roles/d801ebea910445a7a6100b35e07630dd => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:01:50.888880 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5a1a9-5951-4bbe-86bc-433a11a45e09 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:50.895028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5a1a9-5951-4bbe-86bc-433a11a45e09 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:50.895028 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5a1a9-5951-4bbe-86bc-433a11a45e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:50.918545 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1fe5a1a9-5951-4bbe-86bc-433a11a45e09 None None] MFA Rules not processed for user `89243b6377da407288c165f001a14c15`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:50.919753 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1fe5a1a9-5951-4bbe-86bc-433a11a45e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:50.984011 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1471/2927] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:50.994629 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ccfdc-4a0e-43cb-8ea7-c516ad1501c1 admin admin] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:50.996304 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-0b2ccfdc-4a0e-43cb-8ea7-c516ad1501c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:51.031763 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:51.054760 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ccfdc-4a0e-43cb-8ea7-c516ad1501c1 admin admin] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:51.058592 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ea266-8753-42e5-8747-7820dfdfe391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '727f7207d82640a3a228dfb83814148b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:51.059221 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ea266-8753-42e5-8747-7820dfdfe391 service cinder] REQUEST_METHOD: `GET` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:51.059499 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ea266-8753-42e5-8747-7820dfdfe391 service cinder] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:51.059814 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ea266-8753-42e5-8747-7820dfdfe391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:51.068078 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ea266-8753-42e5-8747-7820dfdfe391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:51.070005 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ea266-8753-42e5-8747-7820dfdfe391 service cinder] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:51.078845 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1457/2928] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:01:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:01:54.997752 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5a1a9-5951-4bbe-86bc-433a11a45e09 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:55.018600 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:01:55.064407 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5a1a9-5951-4bbe-86bc-433a11a45e09 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:55.069438 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:55.070102 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:55.070383 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:55.070658 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] PATH_INFO: `/v3/users/89243b6377da407288c165f001a14c15` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:55.071906 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] RBAC: Authorizing `identity:delete_user(user_id=89243b6377da407288c165f001a14c15)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:55.073849 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:55.223749 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89243b6377da407288c165f001a14c15'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:55.236416 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89243b6377da407288c165f001a14c15'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:55.289114 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89243b6377da407288c165f001a14c15'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:55.289805 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89243b6377da407288c165f001a14c15'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:55.297013 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89243b6377da407288c165f001a14c15'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:55.303005 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1472/2929] 10.222.0.42 () {58 vars in 1224 bytes} [Sat Sep 17 04:01:54 2022] DELETE /identity/v3/users/89243b6377da407288c165f001a14c15 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 04:01:55.307759 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dab4aa-7e3e-4fc3-9557-1f4126dc5fde None None] REQUEST_METHOD: `POST` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:55.308258 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dab4aa-7e3e-4fc3-9557-1f4126dc5fde None None] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:55.308363 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dab4aa-7e3e-4fc3-9557-1f4126dc5fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:55.327555 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.auth.core [None req-41dab4aa-7e3e-4fc3-9557-1f4126dc5fde None None] MFA Rules not processed for user `bb19e7f52e3f4b85bcc7952e23773462`. Rule list: `[]` (Enabled: `True`). {{(pid=81726) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 04:01:55.328654 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-41dab4aa-7e3e-4fc3-9557-1f4126dc5fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:55.387503 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1458/2930] 10.222.0.42 () {58 vars in 958 bytes} [Sat Sep 17 04:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 04:01:55.399522 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:55.400987 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:55.457619 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb04f62-9b32-47ff-a1ee-b6a46863238e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:55.461903 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d853e7b-bcd3-4134-b788-4e2fd54f24ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6336f0c469b84db78a90cf8435c7dbe4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:55.462494 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d853e7b-bcd3-4134-b788-4e2fd54f24ab service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:55.462769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d853e7b-bcd3-4134-b788-4e2fd54f24ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:55.463041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d853e7b-bcd3-4134-b788-4e2fd54f24ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:55.471796 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d853e7b-bcd3-4134-b788-4e2fd54f24ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:55.473205 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d853e7b-bcd3-4134-b788-4e2fd54f24ab service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:55.482769 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1473/2931] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:01:55.806348 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dab4aa-7e3e-4fc3-9557-1f4126dc5fde None None] Authenticating user token {{(pid=81726) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:01:55.807793 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.fernet_utils [None req-41dab4aa-7e3e-4fc3-9557-1f4126dc5fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81726) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:01:55.820189 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dab4aa-7e3e-4fc3-9557-1f4126dc5fde None None] Validating token access rules against request {{(pid=81726) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:01:55.825274 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb19e7f52e3f4b85bcc7952e23773462', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545b5824a9314a3a81e4babed059fa06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81726) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:01:55.825985 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:01:55.826265 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:01:55.826532 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] PATH_INFO: `/v3/projects/0bf7d18b6eae44589fc5ff45b4e1fd93` {{(pid=81726) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:01:55.828314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bf7d18b6eae44589fc5ff45b4e1fd93)` {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:01:55.830567 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] RBAC: Authorization granted {{(pid=81726) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:01:55.924314 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bf7d18b6eae44589fc5ff45b4e1fd93', 'request_id': 'req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:55.991949 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bf7d18b6eae44589fc5ff45b4e1fd93', 'request_id': 'req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:55.998101 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bf7d18b6eae44589fc5ff45b4e1fd93', 'request_id': 'req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a'} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:56.009691 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] The token cache is being invalidate because project 0bf7d18b6eae44589fc5ff45b4e1fd93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81726) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 04:01:56.010008 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:56.010638 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG keystone.notifications [None req-6e3ae3b7-2f16-444a-a42e-9d9b48faf89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81726) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 04:01:56.012109 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: [pid: 81726|app: 0|req: 1459/2932] 10.222.0.42 () {58 vars in 1233 bytes} [Sat Sep 17 04:01:55 2022] DELETE /identity/v3/projects/0bf7d18b6eae44589fc5ff45b4e1fd93 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 04:02:05.478682 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=70,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:02:06.006898 dv-cinder-849143-3-iscsi devstack@keystone.service[81726]: DEBUG dbcounter [-] [81726] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:02:59.362616 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d853e7b-bcd3-4134-b788-4e2fd54f24ab service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 04:02:59.364010 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9d853e7b-bcd3-4134-b788-4e2fd54f24ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:02:59.433692 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d853e7b-bcd3-4134-b788-4e2fd54f24ab service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 04:02:59.495321 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165afb54-4a2b-4f0a-bdfe-90248125a4ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ec5c9c71d14762a4869b7d66e87ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bef114f2baf2415ba9942c8358a243db', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 04:02:59.496041 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165afb54-4a2b-4f0a-bdfe-90248125a4ba service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 04:02:59.496324 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165afb54-4a2b-4f0a-bdfe-90248125a4ba service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 04:02:59.496592 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165afb54-4a2b-4f0a-bdfe-90248125a4ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 04:02:59.498330 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-165afb54-4a2b-4f0a-bdfe-90248125a4ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 04:02:59.509458 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165afb54-4a2b-4f0a-bdfe-90248125a4ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 04:02:59.511047 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165afb54-4a2b-4f0a-bdfe-90248125a4ba service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 04:02:59.521008 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1474/2933] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Sep 17 04:02:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 04:03:09.516861 dv-cinder-849143-3-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=58 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}